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1.  Introduction 


1.1  Statement  of  the  Problem 

Scalability  has  been  given  many  different  definitions,  depending  upon  the  background  of  the 
person  defining  it,  the  technology  being  considered,  and  the  operational  use  of  the  technology. 
Typically,  when  interface  designers  talk  about  scalable  interfaces  they  are  referring  to  a  design 
that  ensures  that  development  takes  into  account  the  requirement  to  change  over  time.  This 
drives  interfaces  to  be  flexible  and  “future  proof’.  Without  this  type  of  scalability,  interface 
designs  require  a  complete  renovation  when  a  small  change  is  needed  in  the  application.  This 
is  a  broad  definition  of  scalability  and  has  application  to  robotic  interfaces.  For  this  experiment, 
we  concentrated  on  a  more  narrow  definition  of  scalability  which  is  a  component  of  the  broader 
definition.  We  are  concerned  with  the  ability  of  interfaces  to  accommodate  presentation  on 
devices  of  different  types  and  sizes  as  effectively  as  possible.  Soldiers  operate  in  a  large  range 
of  environments,  from  the  relatively  stable  and  spacious  environment  of  a  tactical  operations 
center  (TOC)  to  the  cramped  and  perpetual  motion  environment  of  a  vehicle  to  the  rugged  and 
physically  demanding  environment  of  the  dismounted  Soldier.  All  these  environments  have  an 
impact  on  the  size  and  configuration  of  the  robotic  interface.  It  is  easy  to  see  that  a  dismounted 
Soldier  cannot  carry  the  relatively  large  controller  that  can  be  used  in  a  TOC.  This  type  of 
scalability  of  interfaces  is  very  important  because  it  ensures  that  training  transfer  is  easy  across 
environments  and  that  interfaces  can  be  tailored  to  the  environment  in  which  they  are  used. 

Interface  trends  are  moving  away  from  “one  size  fits  all”  toward  a  scalable  family  of  products 
with  common  architecture,  but  sizes  depend  on  the  role  and  mission.  The  key  to  ensuring  that  a 
system  is  scalable  is  to  consider  not  only  the  range  of  devices  that  Soldiers  will  use  but  also  their 
context  of  use.  An  example  can  be  easily  seen  in  e-mail  access.  Typically,  individuals  have 
used  their  desktop  computers  to  access  e-mail,  but  more  frequently,  they  are  now  using  personal 
digital  assistants  (PDAs)  and  cell  phones  to  do  the  same  job  when  they  are  outside  their  offices. 
One  factor  affecting  scalability  of  displays  to  the  environment  is  screen  size.  Designing  for  one 
“optimum”  screen  size  may  seem  to  be  a  good  idea  because  it  may  seem  that  the  presentation  of 
the  interface  is  being  controlled.  If  a  display  is  designed  that  only  works  on  one  “fixed”  browser 
window  size,  it  will  not  work  well  in  the  others.  It  might  even  be  completely  unusable  if 
important  features  disappear  off  the  edge  of  the  screen. 

Many  current  robots  (i.e.,  the  MATILDA2,  PackBot3,  and  TALON4)  provide  a  tele-operation 
interface  that  is  large  and  heavy.  A  goal  of  Program  Executive  Office  (PEO)  Soldier  Warrior  is  to 


2Mesa  Associates’  Tactical  Integrated  Light-Force  Deployment  Assembly 

3PackBot  is  a  registered  trademark  of  iRobot  Corporation. 

4TALON,  which  is  a  registered  trademark  of  Foster-Miller,  Inc.,  is  not  an  acronym. 
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design  and  build  an  innovative  universal  robot  controller  that  allows  a  dismounted  Soldier  to 
control  and  task  various  small  robotic  platforms  without  causing  unnecessary  additional  weight 
and  without  bulky  “add-ons”  to  the  Infantry  Soldier  System.  Since  current  robotic  interfaces 
would  add  much  weight  to  the  dismounted  Soldier,  it  is  important  to  discover  and  document  the 
range  of  interface  sizes  that  can  be  used  for  robotic  operations  in  different  environments  and  to 
understand  the  trade-offs  involved  in  tailoring  the  sizes  to  the  environments  in  which  they  will  be 
used. 

The  popularity  of  very  large  screen  displays  and  small,  portable,  wearable  computing  devices  is 
increasing.  The  motivation  behind  the  development  for  many  of  the  large  screen  displays  is  often 
to  provide  an  immersive  experience  or  a  sense  of  presence  for  virtual  reality  and  home  theaters. 
There  are  several  configurations  for  these  displays  (i.e.,  projection  screen,  liquid  crystal  display 
[LCD],  plasma,  etc.),  and  the  prices  for  these  systems  are  dropping.  The  use  of  large  displays  for 
robotic  interfaces  might  allow  a  more  immersive  experience  for  the  operator  than  is  allowed  by  a 
smaller  desktop  size  display  (Tan,  Gergle,  Scupelli,  &  Pausch,  2003).  Tan,  Robertson,  and 
Czerwinski  (2001)  found  that  the  wide  fields  of  view  afforded  by  large  displays  also  provided 
better  cues  to  aid  users  in  navigating  virtual  space.  Although  this  avenue  of  investigation  is  an 
interesting  one,  the  number  of  feasible  display  sizes  (large  and  small)  is  too  large  to  address  in  one 
experiment.  This  experiment  focuses  on  the  PEO  Soldier  goal  of  reducing  the  size  of  the  interface 
for  use  by  dismounted  Soldiers  in  real-world  environments  and  on  the  task  of  driving  a  well- 
marked  course  while  situational  awareness  (SA)  is  maintained  by  the  search  for  targets  in  the 
immediate  area.  In  order  to  bound  even  the  number  of  sizes  of  smaller  displays,  we  focused  on 
display  sizes  that  are  currently  available  (or  will  be  available  in  the  near  future)  to  the  dismounted 
Soldier.  The  thought  here  is  that  a  Soldier  might  not  have  to  carry  a  separate  display  for  robotic 
control,  but  rather,  s/he  could  control  a  robot  using  a  display  that  s/he  is  already  carrying.  The 
four  display  sizes  we  investigated  were  the  Force  XXI  Battle  Command  Brigade  and  Below 
(FBCB2)  that  is  currently  available  in  vehicles,  the  commander’s  digital  assistant  (CD A)  that  is 
being  carried  by  lower  echelon  commanders,  a  PDA  that  is  being  considered  for  squad  members 
by  the  Future  Force  Warrior  (FFW)  program,  and  a  goggle-mounted  display  (GMD)  that  is  being 
considered  for  squad  leaders  by  the  FFW  program. 

1.1.1  The  FBCB2  Display 

The  FBCB2  forms  the  principal  digital  command  and  control  (C2)  system  for  the  Army  at  brigade 
levels  and  below.  It  is  a  7.3-pound,  2.36-inch  by  13.1-inch  by  9-inch  high-resolution  active  matrix 
touch  screen  display  (see  figure  1).  The  screen  is  a  super  video  graphics  array  12.1  inches 
diagonal  with  800x600-pixel  resolution. 

1.1.2  CDA  Display 

The  CDA  is  an  early  “spiral-out”  from  Land  Warrior  that  provides  a  tactical  picture  to  company- 
level  leaders  and  above.  There  is  more  than  one  model  that  uses  the  name  CDA.  The  CDA, 
developed  from  Raytheon’s  air  warrior  digital  kneeboard,  measures  approximately  7  by  10  by  2 
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inches  and  weighs  5.4  lb,  including  its  battery.  It  has  a  6.4-inch  diagonal  daylight  readable 
480x640  resolution  color  LCD  with  an  integrated  touch  screen  (see  figure  2). 


Figure  1.  FBCB2  display. 


Figure  2.  CDA  display. 
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1.1.3  PDAs 


PDAs  are  potentially  attractive  interfaces  for  dismounted  Soldiers  because  they  are  relatively  inex¬ 
pensive,  lightweight,  small,  and  extremely  portable,  and  some  feature  touch-sensitive  displays. 
Many  standard  PDA  interfaces  have  been  developed  for  a  wide  range  of  applications  such  as  word 
processing,  calendar  management,  calculators,  educational  tools,  and  mobile  surveillance.  The 
FFW  digital  assistants  also  display  navigation  and  SA  pictures  and  images  received  from  team 
members.  The  Recon  400  X  used  by  the  FFW  Soldiers  during  the  Air  Assault  Expeditionary  Force 
(AAEF)  Spiral  C  had  a  display  resolution  of  240x320  pixels,  a  3.5-inch  diagonal  display  size,  and 
weighed  17  ounces  (see  figure  3). 


Figure  3.  Vehicle  target  detection  rates 
by  distance,  open  field  target 
detection,  no  smoke. 


1.1.4  Head-Mounted  Displays  (HMDs)  and  GMDs 

Small,  wearable  HMDS  and  GMDs  are  being  developed  that  enable  users  to  observe  a  high- 
resolution  display  without  having  to  carry  a  bulky  display  or  without  restricting  the  user  to  small 
size  and  low  resolution.  These  devices  come  in  a  variety  of  configurations  (monocular,  binocular, 
see-through,  opaque,  etc.).  Some  are  mounted  on  straps  worn  around  the  head,  some  are  mounted 
on  helmets,  and  some  are  mounted  on  eyewear.  The  FFW  program  is  investigating  several  of 
these  displays  for  use  by  the  squad  and  team  leaders.  Typically,  these  devices  provide  lightweight 
(~17  ounces)  super-video  graphics  display,  high  resolution  (800x600)  pictures  with  a  1.425-inch 
diagonal  picture.  The  device  used  in  this  study  was  a  monocular  GMD.  Because  this  display  was 
so  close  to  the  eye,  its  apparent  size  was  like  that  of  a  17-inch  diagonal  TV  display  (see  figure  4). 
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Figure  4.  GMD. 


1.2  Objectives 

The  objective  of  this  experiment  was  to  determine  what  effect  display  size  reduction  has  on  the 
tele-operation  (driving)  of  small  robots  and  the  operator’s  SA  of  the  immediate  vicinity  of  the 
robot. 


2.  Method 


2.1  Participants 

Thirty-two  Soldiers  from  the  Officer  Candidate  School  (OCS),  Fort  Benning,  Georgia,  participated 
in  the  study.  These  Soldiers  had  experience  as  enlisted  Soldiers  and  came  from  varied  military 
occupational  specialties  (MOSs). 

2.2  Instruments  and  Apparatus 
2.2.1  TALON  Robot 

The  TALON  is  a  lightweight  robot  designed  for  missions  ranging  from  reconnaissance  to  weapons 
delivery  (see  figure  5).  Built  with  all-weather,  day/night  and  amphibious  capabilities,  the  TALON 
can  operate  during  adverse  conditions  over  almost  any  terrain.  The  suitcase-portable  robot  is  con¬ 
trolled  through  a  two-way  radio  frequency  line  from  a  portable  operator  control  unit  that  provides 
continuous  data  and  video  feedback  for  precise  vehicle  positioning.  It  was  developed  for  the 
Explosive  Ordnance  Disposal  Technology  Directorate  of  the  U.S.  Army’s  Armament  Research, 
Development,  and  Engineering  Center  at  Picatinny  Arsenal,  New  Jersey,  by  the  engineering  and 
technology  development  firm  of  Foster-Miller.  The  TALON  began  being  used  in  military  opera¬ 
tions  in  Bosnia  in  2000,  deployed  to  Afghanistan  in  early  2002,  and  has  been  in  Iraq  since  the  war 
started,  assisting  with  improvised  explosive  device  (IED)  detection  and  removal. 
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Figure  5.  TALON  robot. 


For  this  experiment,  the  TALON  was  equipped  with  a  video  camera  that  enabled  the  participants 
to  maneuver  the  vehicle  and  assess  enemy  activity  and  IEDs  along  the  road  to  the  objective. 

2.2.2  Robotic  Vehicle  Displays 

Four  different  display  sizes  were  used  to  conduct  this  experiment.  The  A,  B,  and  C  display 
configurations  were  presented  on  an  iExplorer  to  control  the  number  of  variables  present  in  the 
study.  Both  the  iExplorer  (see  figure  6)  and  the  GMD  (see  figure  7)  were  plugged  into  the  existing 
TALON  control  system  so  that  its  radio  could  be  used.  However,  the  TALON  control  system 
joystick  and  display  were  not  used. 


Figure  6.  The  iExplorer  display  and  TALON  radio  configuration. 
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Figure  7.  The  GMD  and  TALON  radio  configuration. 


The  four  display  configurations  (sizes  or  types)  were  representative  of  displays  that  can  be  used 
in  the  field  or  that  may  soon  be  present.  The  display  configurations  used  were 

•  Display  A  -  A  display  based  on  the  Stryker  and  Bradley  FBCB2  display  characteristics  (a 
10.4-inch  diagonal  screen  with  800x600  pixels); 

•  Display  B  -  A  display  based  upon  the  CD  A  display  characteristics  (a  6. 5 -inch  diagonal 
screen  with  640x480  pixels); 

•  Display  C  -  A  display  based  upon  the  FFW  PDA  display  characteristics  (a  3. 5 -inch 
diagonal  screen  with  240x320  pixels);  and 

•  Display  D  -  A  GMD  display  based  on  the  FFW  GMD  display  characteristics  (a  1.425-inch 
diagonal  screen  with  800x600  pixels). 

2.2.3  Robotic  Driving  Course 

The  robotic  driving  course  consisted  of  an  oval-shaped  course  with  four  different  lanes.  Each 
lane  had  three  legs  and  a  total  length  of  approximately  300  meters  (see  figure  8).  Leg  A,  the  first 
leg  of  each  lane  (see  figure  9),  was  marked  with  engineering  tape  and  required  the  Soldier  to 
drive  as  quickly  as  possible  to  the  end  of  the  leg.  An  obstacle  placed  on  the  path  was  situated  at 
the  end  of  the  first  leg.  For  Leg  B,  the  operator  was  required  to  negotiate  around  the  obstacle  by 
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going  off  the  path  and  then  returning  to  the  path  by  the  shortest  and  easiest  route  (see  figure  10). 
Mock-up  enemy  Soldiers,  booby  traps,  IEDs,  and  mines  (see  figure  1 1)  that  could  be  clearly  seen 
by  the  driving  camera  were  placed  along  the  rest  of  the  lane  (Leg  C)  between  the  obstacle  and 
the  objective  (end  of  the  course).  The  robotic  operators  tele-operated  the  TALON  from  inside  a 
tent  that  was  placed  behind  a  benn  that  blocked  the  line  of  sight  (LOS)  between  the  operators 
and  the  course.  The  tent  and  its  placement  prevented  the  operator  from  tele-operating  the  vehicle 
by  using  LOS  rather  than  the  display.  It  also  kept  the  operator  and  the  equipment  out  of  the 
elements. 


Figure  9.  Leg  A,  maneuver. 
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Figure  10.  Leg  B,  obstacle  avoidance. 


Figure  11.  Leg  C,  target  detection  and  identification. 

2.2.4  Questionnaires 

Questionnaires  elicited  Soldiers’  opinions  about  their  performance  and  experiences  with  each  of 
the  display  systems.  The  questionnaires  asked  the  Soldiers  to  rate  the  devices  on  a  7-point  Likert 
scale  ranging  from  “extremely  good/easy”  to  “extremely  bad/difficulf Questionnaires  were 
administered  to  each  Soldier  at  the  completion  of  the  training  course,  at  the  end  of  each  iteration 
with  the  displays,  and  at  the  end  of  the  experiment. 
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2.2.5  National  Aeronautics  and  Space  Administration  (NASA)  Task  Load  Index  (TLX) 

The  NASA  TLX  is  a  subjective  workload  assessment  tool  that  allows  subjective  workload 
assessments  on  operator(s)  working  with  various  human-machine  systems  (Hart  &  Staveland, 
1988).  It  uses  a  multi-dimensional  rating  procedure  that  derives  an  overall  workload  score  based 
on  a  weighted  average  of  ratings  on  six  subscales.  These  subscales  include  mental  demands, 
physical  demands,  temporal  demands,  own  performance,  effort,  and  frustration.  It  can  be  used  to 
assess  workload  in  various  human-machine  environments  such  as  aircraft  cockpits;  command, 
control,  and  communication  workstations;  supervisory  and  process  control  environments; 
simulations;  and  laboratory  tests. 

2.3  Procedures 

2.3.1  Training 

The  32  experiment  Soldiers  reported  in  groups  of  four  for  one  day  each.  Before  the  first  training 
presentation,  experiment  Soldiers  received  a  roster  number,  which  was  used  to  identify  them 
throughout  the  evaluation.  They  were  given  an  operations  order  that  explained  the  robotic  mission 
that  they  would  undertake  during  the  experiment.  Driving  requirements,  obstacle  negotiation 
requirements,  and  identification  of  threats  were  all  explained  and  any  questions  the  Soldiers  had 
concerning  the  experiment  were  answered.  Soldiers  were  then  shown  examples  of  the  targets  and 
the  IEDs  that  they  would  be  required  to  identity  as  they  negotiated  the  robotic  course.  A  represen¬ 
tative  from  the  Space  and  Naval  Warfare  Systems  Command  trained  the  Soldiers  in  the  use  of  the 
TALON  robot.  Soldiers  practiced  tele-operating  the  robot  on  a  training  course  that  was  similar  to 
the  actual  course  used  during  the  experiment.  This  course  required  driving,  obstacle  negotiation, 
and  identification  of  threats.  Soldiers  were  trained  with  the  actual  TALON  display  rather  than  any 
of  the  experimental  displays.  Upon  completion  of  the  course,  the  Soldiers  were  given  a  question¬ 
naire  designed  to  assess  their  perception  of  the  training  adequacy.  Questions  about  the  amount  of 
practice  time  given,  the  level  of  detail  presented,  and  the  adequacy  of  training  aids  were  asked. 

2.3.2  Demographics 

Demographic  data,  as  well  as  data  concerning  the  Soldiers’  Army  and  robotic  experience,  were 
documented  for  each  Soldier. 

2.3.3  Tele-operation  Trials 

Soldiers  were  assigned  to  displays  and  lanes  according  to  the  matrix  presented  in  table  1 .  The 
matrix  was  constructed  with  the  use  of  a  4x4  Greco-Williams  square  for  treatment  conditions, 
lanes,  and  preceding  device  (so  that  no  device  always  followed  another). 

The  Soldiers  were  required  to  tele-operate  the  robot  within  the  course  as  quickly  and  efficiently  as 
possible,  negotiate  around  obstacles  in  the  path,  and  detect  and  identity  silhouette  targets,  IEDs, 
mines,  and  booby  traps  on  the  course.  A  data  collector  walked  behind  the  TALON  on  each  trial, 
documenting  the  following  performance  measures: 
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Time  to  reach  the  end  of  each  leg, 


The  number  of  major  times  the  driver  went  off  course  (i both  wheels  outside  the  marked 
course), 

The  number  of  minor  times  the  driver  went  off  course  ( one  wheel  touched  or  went  outside 
marked  course)  on  the  first  and  third  legs, 

The  number  of  driving  errors  ( back-ups ,  turning  in  the  wrong  direction,  etc)  on  each  leg, 
The  time  to  complete  Leg  A  of  the  course, 

The  time  required  to  re-route  around  terrain  obstacles  (dynamic  retasking)  on  Leg  B, 

The  number  of  enemy  personnel  detected  and  identified  and  the  range  of  detection, 

The  number  of  booby  traps,  IEDs,  and  mines  detected  and  identified  and  the  range  of 
detection, 

The  number  of  false  threats  found, 

The  time  to  complete  Leg  C  of  the  course,  and 
The  overall  course  completion  time. 


Table  1.  Order  of  treatments  and  lanes. 


Roster 

(Soldier) 

Iteration 

1 

2 

3 

4 

Display 

Lane 

Display 

Lane 

Display 

Lane 

Display 

Lane 

1,9,  17,  25 

A 

1 

D 

4 

B 

2 

C 

3 

2,  10,  18,  26 

B 

4 

A 

3 

C 

1 

D 

2 

3,  11,  19,27 

C 

2 

B 

1 

D 

3 

A 

4 

4,  12,  20,  28 

D 

3 

C 

2 

A 

4 

B 

1 

5,  13,21,29 

C 

4 

B 

3 

D 

1 

A 

2 

6,  14,  22,  30 

D 

1 

C 

4 

A 

2 

B 

3 

7,  15,23,31 

A 

3 

D 

2 

B 

4 

C 

1 

8,  16,  24,  32 

B 

2 

A 

1 

C 

3 

D 

4 

The  Soldiers  received  the  following  written  operations  order: 

Who:  You  are  in  2nd  Platoon,  A  company,  1st  Battalion,  52nd  Infantry  Division 

What:  You  are  located  in  a  forward  operating  base  (FOB)  that  is  located  in  a  remote  area  near 
the  capital  of  Iraq.  Your  perimeter  must  constantly  be  checked  for  IEDs  and  enemy  person¬ 
nel  as  your  FOB  has  been  targeted  by  anti-Iraqi  forces  (AIF).  You  have  been  tasked  to 
reconnoiter  the  small  trail  around  your  perimeter  to  ensure  that  there  are  no  IEDs,  unau¬ 
thorized  militia,  etc.,  and  report  any  and  all  such  activity.  In  an  effort  to  avoid  unmanned 
sensors  in  the  area,  there  is  a  well-marked  trail  around  the  perimeter  that  unmanned  ground 
vehicles  (UGVs)  can  use,  and  movement  outside  this  marked  trail  is  not  authorized.  Con¬ 
certina  wire  at  different  points  around  the  perimeter  must  be  avoided.  The  UGV  route  is 
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clearly  marked  up  to  the  obstacle,  and  the  marked  lane  continues  on  the  other  side  of  the 
obstacle.  Obstacles  can  be  traversed  either  to  the  left  or  right,  depending  on  the  obstacle; 
however,  the  route  around  the  obstacle  is  not  marked. 

When:  The  time  is  the  local  time. 

They  were  told  to  reconnoiter  each  leg  of  the  route  and  to  find  as  many  of  the  targets  as  possible  as 
quickly  as  possible.  They  were  then  instructed  that  they  must  be  at  the  lane  end  point  no  later  than 
30  minutes  after  starting  the  course.  They  indicated  each  target  to  the  data  collector  following  the 
TALON  who  used  a  hand-held  laser  range  finder  to  detennine  the  detection  distance  and  reported 
the  distance  to  another  data  collector  in  the  tent.  Each  target  was  labeled  with  a  number.  The 
trailing  data  collector  radioed  the  target  number  that  was  found  and  the  distance  to  the  target  to  the 
recording  data  collector.  He  also  radioed  the  number  and  type  of  driving  errors,  the  number  of 
times  the  Soldier  went  off  course,  the  time  for  completion  of  each  leg  of  the  course,  and  the  total 
course  time. 

The  Soldiers  tele-operated  the  robot  using  the  four  different  display  configurations  (one  on  each 
of  the  four  different  course  lanes).  After  each  lane,  the  Soldier  responded  to  questionnaire  items 
concerning  the  display  that  was  used  and  waited  until  the  course  was  ready  for  the  next  iteration. 
At  the  end  of  the  day,  each  Soldier  completed  a  questionnaire  concerning  his  or  her  overall 
assessment  of  each  display  configuration. 


3.  Results 


3.1  Training 

Soldiers  rated  the  training  course  highly  and  stated  that  they  had  enough  training  time  to  feel 
confident  that  they  could  perform  well  on  the  course. 

3.2  Demographics 

The  32  OCS  Soldiers  who  participated  in  this  experiment  averaged  8  years  3  months  in  the 
military.  Twelve  Soldiers  had  served  in  an  infantry  MOS.  Seventeen  of  the  Soldiers  had  been 
deployed  in  a  combat  area.  Although  only  one  had  robotic  experience,  27  Soldiers  had  driven  a 
remote  control  car.  The  weights  of  the  Soldiers  ranged  from  the  40th  percentile  female  to  the  99th 
percentile  male  (130  to  260  pounds).  Their  heights  ranged  from  the  5th  percentile  female  to  the 
99th  percentile  male  (60  to  80  inches).  The  average  age  of  these  Soldiers  was  3 1  years.  Eight 
Soldiers  were  left-eye  dominant  and  seven  wore  prescription  lenses.  All  but  one  Soldier  had  at 
least  20/20  corrected  vision  in  one  eye.  The  one  exception  had  20/25  vision  in  both  eyes.  See 
appendix  A  for  detailed  demographic  infonnation.  Detailed  results  are  presented  in  appendix  A. 
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3.3  Tele-operation  Trials 
3.3.1  Course  Completion  Times 

Mean  times  to  complete  each  leg  of  the  course  and  the  entire  course  are  shown  in  table  2.  A  series 
of  repeated  measures  analyses  of  variances  (ANOVAs),  summarized  in  table  3,  indicate  that  there 
was  a  significant  difference  among  the  display  conditions  for  the  time  required  to  complete  Leg  B 
of  the  course.  Figure  12  illustrates  this  finding.  Follow-on  pair-wise  comparisons  were  conducted 
with  the  use  of  Holm’s  Sequential  Bonferroni  to  control  family-wise  error  rates.  As  shown  in 
table  4,  Leg  B  completion  times  were  significantly  longer  (slower)  with  the  D  display  than  with 
the  B  and  C  displays. 


Table  2.  Time  (min:sec)  to  complete  course  legs. 


Leg  A 

Leg  B 

Leg  C 

Total  Course 

Display 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

A 

1:53 

0:32 

1:44 

1:14 

12:09 

4:21 

15:45 

4:44 

B 

1:49 

0:37 

1:13 

0:36 

11:14 

3:10 

14:16 

3:40 

C 

1:47 

0:34 

1:21 

0:38 

11:49 

3:40 

14:57 

3:54 

D 

2:08 

0:51 

1:47 

0:52 

11:43 

3:56 

15:39 

4:26 

Table  3.  Summary  of  ANOVAs,  leg  and  course  completion  times. 


Leg 

F 

df 

P 

»i2p 

A 

1.97 

3,93 

0.123 

0.060 

B 

4.49 

3,93 

0.005* 

0.127 

C 

0.62 

3,93 

0.602 

0.020 

Total 

1.55 

3,93 

0.207 

0.48 

*p<  .05 


Figure  12.  Mean  times  for  each  display  on  leg  B  (minutes). 
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Table  4.  Follow-on  pair-wise  comparisons,  leg  B  course  completion  times. 


Display  Pair 

t 

Df 

obtained  p 

required  p 

A  vs.  B 

2.16 

31 

0.039 

0.0125 

A  vs.  C 

1.58 

31 

0.125 

0.0167 

A  vs.  D 

-0.27 

31 

0.788 

0.05 

B  vs.  C 

-1.34 

31 

0.189 

0.025 

B  vs.  D 

-4.40 

31 

<.001* 

0.0083 

C  vs.  D 

-3.48 

31 

.002* 

0.01 

* p  <  .05,  2  tailed 


3.3.2  Driving  Errors 

Table  5  shows  the  mean  number  of  driving  errors  with  each  display  size  for  the  three  legs. 
Repeated  measures  ANOVAs  were  conducted  to  test  for  differences  among  the  means  within 
each  leg  of  the  course.  As  shown  in  table  6,  none  of  the  ANOVAs  yielded  statistically 
significant  results. 


Table  5.  Driving  errors  by  leg. 


Leg  A 

Leg  B 

Leg  C 

Display 

Mean 

SD 

Mean 

SD 

Mean 

SD 

A 

0.00 

0.00 

0.19 

0.54 

0.25 

0.72 

B 

0.09 

0.30 

0.03 

0.18 

0.13 

0.71 

C 

0.06 

0.35 

0.09 

0.30 

0.34 

1.21 

D 

0.03 

0.18 

0.28 

0.68 

0.25 

0.80 

Table  6.  Summary  of  ANOVAs,  driving  errors. 


Leg 

F 

df 

P 

n2,, 

A 

0.83 

3,93 

0.481 

0.026 

B 

1.77 

3,93 

0.158 

0.054 

C 

0.27 

3,93 

0.707 

0.015 

The  mean  numbers  of  major  off-course  events  for  each  leg  and  each  display  condition  are  shown 
in  table  7.  A  series  of  repeated  measures  ANOVAs,  summarized  in  table  8,  indicates  that  none 
of  the  differences  among  the  means  was  statistically  significant. 


Table  7.  Major  off-course  errors. 


Leg  A 

Leg  B 

Leg  C 

Display 

Mean 

SD 

Mean 

SD 

Mean 

SD 

A 

0.09 

0.30 

0.38 

0.61 

0.03 

0.18 

B 

0.09 

0.39 

0.25 

0.57 

0.06 

0.25 

C 

0.03 

0.18 

0.19 

0.47 

0.03 

0.18 

D 

0.16 

0.45 

0.22 

0.49 

0.09 

0.30 

14 


Table  8.  Summary  of  ANOVAs,  major  off-course  errors. 


Leg 

F 

df 

P 

»12p 

A 

0.69 

3,93 

0.561 

0.022 

B 

0.87 

3,93 

0.460 

0.270 

C 

0.57 

3,93 

0.635 

0.018 

Table  9  shows  the  average  number  of  minor  off-course  errors  on  each  leg  of  the  course  with  each 
display.  Repeated  measures  ANOVAs  (table  10)  indicate  that  none  of  the  differences  among  the 
means  was  statistically  significant. 

Table  9.  Minor  off-course  errors. 


Leg  A 

Leg  B 

Display 

Mean 

SD 

Mean 

SD 

A 

0.16 

0.51 

0.91 

1.23 

B 

0.25 

0.67 

0.81 

1.12 

C 

0.19 

0.47 

1.59 

2.34 

D 

0.38 

0.61 

1.16 

1.63 

Table  10.  Summary  of  ANOVAs,  minor  off-course  errors. 


Leg 

F 

df 

P 

n2„ 

A 

0.965 

3,93 

0.413 

0.03 

C 

1.78 

3,93 

0.157 

0.054 

3.3.3  Target  Detection  and  Identification 

Table  1 1  shows  the  mean  number  of  three-dimensional  (3-D)  silhouette  targets  detected  and 
correctly  identified  for  each  display  condition.  Repeated  measures  ANOVAs  (table  12)  indicate 
that  there  were  no  significant  differences  among  the  means  for  target  detection  or  identification. 

Table  11.  Mean  3-D  silhouette  targets  detected  and  identified. 


Display 

Detect 

Identify 

Mean 

SD 

Mean 

SD 

A 

8.41 

1.43 

8.16 

1.46 

B 

8.72 

1.17 

8.38 

1.24 

C 

8.69 

1.47 

8.44 

1.58 

D 

8.28 

1.65 

7.94 

1.70 

15 


Table  12.  Summary  of  ANOVAs,  3-D  silhouette  targets. 


Dependent  variable 

F 

df 

P 

n2P 

Target  detection 

0.86 

3,93 

0.463 

0.027 

Target  identification 

0.97 

3,93 

0.411 

0.030 

The  mean  number  of  IEDs  detected  and  correctly  identified  is  shown  in  table  13.  The  repeated 
measures  ANOVAs  summarized  in  table  14  indicate  that  there  were  no  significant  differences 
among  the  display  conditions  for  target  detection  or  identification. 

Table  13.  Mean  IED  targets  detected  and  identified. 


Display 

Detect 

Identify 

Mean 

SD 

Mean 

SD 

A 

4.66 

1.70 

4.09 

1.67 

B 

5.03 

2.16 

4.38 

2.00 

C 

5.31 

1.94 

4.81 

1.89 

D 

4.41 

2.27 

3.91 

2.10 

Table  14.  Summary  of  ANOVAs,  IED  targets. 


Dependent  variable 

F 

df 

P 

i2p 

Target  detection 

2.20 

3,93 

0.093 

0.066 

Target  identification 

2.26 

3,93 

0.086 

0.068 

The  mean  ranges  to  detect  targets  within  each  display  condition  are  shown  in  table  15.  There 
were  no  significant  differences  among  the  displays  for  either  type  of  target  (table  16). 

Table  15.  Mean  range  (meters)  to  detect  targets. 


Display 

3-D  silhouettes 

IEDs 

Mean 

SD 

Mean 

SD 

A 

9.49 

2.88 

2.55 

1.13 

B 

10.25 

3.58 

2.86 

1.45 

C 

9.16 

2.75 

2.67 

D 

9.07 

3.20 

2.65 

1.52 

Table  16.  Summary  of  ANOVAs,  target  detection  ranges. 


Type  of  Target 

F 

df 

P 

n2P 

3-D  silhouettes 

1.32 

3,93 

0.271 

0.041 

IEDs 

0.49 

3,93 

0.688 

0.016 

Finally,  the  mean  numbers  of  false  target  detections  for  each  display  condition  are  shown  in  table 
17.  A  repeated  measures  ANOVA  indicates  no  significant  difference  among  the  means:  F(3,93) 
=  0.69, p  =  .562,  rfp  =  .033. 
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Table  17.  Mean  number  of  false  target  detections. 


Display 

Mean 

SD 

A 

0.31 

0.69 

B 

0.47 

0.80 

C 

0.28 

0.68 

D 

0.44 

0.84 

3.3.4  Soldier  Evaluation  of  Performance 

Soldiers  indicated  that  all  display  sizes  were  sufficient  for  driving  and  for  providing  adequate  SA 
of  the  immediate  surroundings  for  effective  driving.  Their  mean  subjective  ratings  for  tele-opera¬ 
tion  tasks  with  the  A,  B,  and  C  display  sizes  were  very  similar.  Although  many  ratings  of  the  D 
display  size  were  also  somewhat  similar  to  the  ratings  for  the  other  display  sizes,  D  was  rated  the 
lowest  for  ease  of  perfonning  every  one  of  the  tele-operation  tasks.  The  tasks  for  which  the  D 
display  size  was  most  dissimilar  from  the  other  display  sizes  are  shown  in  table  18. 

Table  18.  Mean  subjective  ratings  for  tasks  with  dissimilar  ratings. 


Task 

Display  A 

Display  B 

Display  C 

Display  D 

Avoid  obstacles 

5.50 

5.62 

5.63 

4.91 

Localize  the  robotic  vehicle  within  the  environ-ment  so  that 
you  can  tell  where  the  sides  of  the  vehicle  are  located 

5.41 

5.31 

5.66 

4.94 

See  well  enough  to  drive  at  slowest  speeds 

5.47 

5.78 

5.94 

5.41 

See  well  enough  to  drive  at  fastest  speeds 

4.28 

4.63 

4.88 

3.94 

Overall  ability  to  perform  driving  tasks 

5.03 

5.31 

5.63 

4.69 

Several  problems  were  reported  with  tele-operating  the  robot,  which  had  nothing  to  do  with  the 
displays.  These  included  interference  in  the  video  transmission,  screen  black-outs,  video  lag,  and 
freezing  of  the  picture.  Soldiers  rated  their  ability  to  maintain  SA  as  similar  for  each  of  the 
displays.  There  were  several  comments  that  SA  was  degraded  because  the  camera  on  the  robot 
had  no  view  to  the  sides  and  rear  and  because  there  was  no  audio  feed. 

3.3.5  Soldier  Evaluation  of  Individual  System  Characteristics 

Table  19  displays  the  system  characteristics  whose  mean  subjective  ratings  were  most  dissimilar. 
Table  19.  Mean  subjective  ratings  for  system  characteristics  with  dissimilar  ratings. 


System  Characteristics 

Display  A 

Display  B 

Display  C 

Display  D 

Comfort  of  viewing  the  display 

4.81 

5.12 

4.91 

4.00 

Resolution  (clarity) 

3.47 

4.25 

4.75 

3.88 

The  Soldiers  rated  the  D  display  as  the  least  comfortable  to  view.  With  the  A,  B,  and  C  display 
sizes,  the  Soldiers  moved  their  heads  and  changed  their  body  positions  to  achieve  the  desired 
viewing  distance.  They  indicated  that  the  head  positions  did  not  detract  from  their  comfort. 
However,  the  reasons  they  gave  for  rating  the  D  display  as  less  comfortable  were  primarily 
attributable  to  eye  strain  from  viewing  the  display  with  only  one  eye.  One  Soldier  also  complained 
that  he  kept  trying  to  turn  his  head  to  see  farther  to  the  left  or  right  with  device  D,  even  though  this 
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action  had  no  effect  on  the  area  of  camera  regard.  One  Soldier  commented  that  he  had  more 
difficulty  finding  targets  with  the  largest  display  size  (A)  because  he  had  to  keep  his  eyes  moving 
in  order  to  scan  the  entire  width  of  the  display.  Others  complained  that  the  largest  display  just  had 
bigger,  blockier  pixels  and  appeared  to  have  lower  resolution  than  devices  B  and  C.  Several 
Soldiers  stated  that  the  smaller  displays  would  definitely  be  advantageous  for  dismounted  Soldiers. 


3.3.6  Soldier  Ratings  of  Physical  Problems  Experienced  With  the  Display  Sizes 

Table  20  displays  the  number  of  Soldiers  who  experienced  different  types  of  physical  problems 
when  they  wore  each  of  the  display  sizes. 

Table  20.  Number  of  physical  problems  experience  with  each  display  size. 


Display 

Number  of  Responses 

A 

B 

c 

D 

Eye  strain 

6 

4 

2 

21 

Tunnel  vision 

1 

1 

1 

2 

Headaches 

0 

0 

0 

1 

Motion  sickness 

0 

0 

0 

3 

Nausea 

0 

0 

0 

1 

Disorientation 

1 

1 

0 

4 

Dizziness 

0 

0 

0 

1 

Any  other  problems? 

1 

0 

0 

4 

Total 

9 

6 

3 

37 

Eye  strain,  motion  sickness,  headaches,  and  disorientation  were  reported  by  many  more  Soldiers 
with  the  GMD.  In  fact,  66%  of  the  Soldiers  reported  eye  strain  from  using  the  GMD.  Many 
complained  that  they  had  to  keep  the  other  eye  closed  or  cover  it  because  the  intensity  of  the 
disparate  images  coming  into  two  different  eyes  was  overwhelming.  Detailed  questionnaire 
results  are  shown  in  appendix  A. 

3.3.7  Soldier  Subjective  Workload 

The  analysis  of  the  Soldiers’  NASA  TLX  ratings  for  the  effort  required  for  operation  of  each 
display  follows. 

3.3.7. 1  Analysis  of  Total  Workload  Scores 

Table  21  provides  the  mean  NASA  TLX  total  workload  scores  for  each  display  option,  and  table 
22  shows  the  results  of  the  follow-on  comparisons.  Differences  among  means  were  statistically 
significant  (repeated  measures  general  linear  model  [GLM];  L  3,93  =  9.494;  p  =  0.00;  rf  =  0.23). 
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Table  21.  Mean  NASA  TLX  total  workload  scores  by  display  option. 


NASA  TLX  Total  Workload 

Mean 

Std.  Deviation 

N 

Display  A 

41.94 

15.52 

32 

Display  B 

36.43 

16.19 

32 

Display  C 

34.16 

14.53 

32 

Display  D 

49.17 

19.68 

32 

Table  22.  Follow-on  paired  comparison  analyses  for  mean  NASA  TLX  total  workload  scores. 


Paired  Comparisons 

t 

N 

P 

Pair  1 

A  versus  B 

1.933 

31 

.062 

Pair  2 

B  versus  C 

.884 

31 

.383 

Pair  3 

C  versus  D 

-4.056 

31 

.000 

Pair  4 

A  versus  D 

-2.254 

31 

.031 

Pair  5 

A  versus  C 

2.529 

31 

.017 

Pair  6 

B  versus  D 

-4.429 

31 

.000 

3. 3. 7.2  Analysis  of  Mental  Demand  Scores 

Table  23  provides  the  mean  NASA  TLX  weighted  scores  for  mental  demand  for  each  display 
option,  and  table  24  shows  the  results  from  the  follow-on  comparisons.  Differences  among 
means  were  statistically  significant  (repeated  measures  GLM;  F  3,93  =  5.517;  p  =  0.02;  p  = 
0.15). 


Table  23.  Mean  NASA  TLX  mental  demand  scores  by  display  option. 


NASA  TLX  Mental  Demand 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

8.62 

6.27 

32 

DISPLAY  B 

9.46 

8.04 

32 

DISPLAY  C 

9.00 

7.09 

32 

DISPLAY  D 

12.40 

8.87 

32 

Table  24.  Follow-on  paired  comparison  analyses  for  mean  NASA  TLX  weighted  mental 
demand  scores. 


Paired  Comparisons 

t 

N 

P 

Pair  1 

A  versus  B 

1.933 

31 

.062 

Pair  2 

A  versus  C 

.884 

31 

.383 

Pair  3 

A  versus  D 

-4.056 

31 

.000 

Pair  4 

B  versus  C 

-2.254 

31 

.031 

Pair  5 

B  versus  D 

2.529 

31 

.017 

Pair  6 

C  versus  D 

-4.429 

31 

.000 
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3.3.73  Analysis  of  Physical  Demand  Scores 


Table  25  provides  the  mean  NASA  TLX  weighted  scores  for  physical  demand  for  each  display 
option,  and  table  26  shows  the  results  from  the  follow-on  comparisons.  Differences  among  means 
were  statistically  significant  (repeated  measures  GLM;  F  3,93  =  3.437; p  =  0.02;  r\  =  0.10). 


Table  25.  Mean  NASA  TLX  physical  demand  scores  by  display  option. 


NASA  TLX  Physical 

Demand 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

1.18 

3.94 

32 

DISPLAY  B 

.56 

1.39 

32 

DISPLAY  C 

.76 

2.40 

32 

DISPLAY  D 

2.96 

6.51 

32 

Table  26.  Follow-on  paired  comparison  analyses  for  mean  NASA  TLX  weighted  physical 
demand  scores. 


Paired  Comparisons 

t 

N 

P 

Pair  1 

A  versus  B 

.870 

31 

.391 

Pair  2 

A  versus  C 

1.395 

31 

.173 

Pair  3 

A  versus  D 

-1.586 

31 

.123 

Pair  4 

B  versus  C 

-.418 

31 

.679 

Pair  5 

B  versus  D 

-2.384 

31 

.023 

Pair  6 

C  versus  D 

-2.109 

31 

.043 

33.7.4  Analysis  of  Temporal  Demand  Scores 

Table  27  provides  the  mean  NASA  TLX  weighted  scores  for  temporal  demand  for  each  display 
option.  Differences  among  means  were  not  statistically  significant  (repeated  measures  GLM; 

F  3,93  =  1.222;  p  =  0.30;  rf  =  0.04). 


Table  27.  Mean  NASA  TLX  temporal  demand  scores  by  display  option. 


NASA  TLX  Temporal  Demand 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

5.8542 

5.20266 

32 

DISPLAY  B 

5.2104 

4.14990 

32 

DISPLAY  C 

6.0042 

5.27774 

32 

DISPLAY  D 

4.4448 

4.92785 

32 

33.7.5  Analysis  of  Performance  Scores 

Table  28  provides  the  mean  NASA  TLX  weighted  scores  for  self-assessment  of  performance  for 
each  display  option.  Differences  among  means  were  not  statistically  significant  (repeated 
measures  GLM;  F  3,93  =  0.916;  p  =  0.44;  rf  =  0.03). 
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Table  28.  Mean  NASA  TLX  performance  scores  by  display  option. 


NASA  TLX  Performance 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

9.65 

6.89 

32 

DISPLAY  B 

8.35 

6.61 

32 

DISPLAY  C 

7.75 

5.80 

32 

DISPLAY  D 

8.26 

6.21 

32 

3. 3. 7. 6  Analysis  of  Effort  Scores 

Table  29  provides  the  mean  NASA  TLX  weighted  scores  for  effort  for  each  display  option,  and 
table  30  shows  the  results  from  the  follow-on  comparisons.  Differences  among  means  were 
statistically  significant  (repeated  measures  GLM;  F  3,93  =  7.327;  p  =  0.00;  rf  =  0.19). 


Table  29.  Mean  NASA  TLX  effort  scores  by  display  option 


NASA  TLX  Effort 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

7.63 

5.75 

32 

DISPLAY  B 

6.93 

4.62 

32 

DISPLAY  C 

6.71 

5.64 

32 

DISPLAY  D 

10.90 

7.35 

32 

Table  30.  Follow-on  paired  comparison  analyses  for  mean  NASA  TLX  weighted  effort  scores. 


Paired  Comparisons 

t 

N 

Sig.  (2-tailed) 

Pair  1 

A  versus  B 

.673 

31 

.506 

Pair  2 

A  versus  C 

.864 

31 

.394 

Pair  3 

A  versus  D 

-3.021 

31 

.005 

Pair  4 

B  versus  C 

.258 

31 

.798 

Pair  5 

B  versus  D 

-4.025 

31 

.000 

Pair  6 

C  versus  D 

-3.957 

31 

.000 

3. 3. 7. 7  Analysis  of  Frustration  Scores 

Table  3 1  provides  the  mean  NASA  TLX  weighted  scores  for  frustration  for  each  display  option, 
and  table  32  shows  the  results  from  the  follow-on  comparisons.  Differences  among  means  were 
statistically  significant  (repeated  measures  GLM;  F  3,93  =  6.007; p  =  0.00;  r\  =  0.16). 


Table  31.  Mean  NASA  TLX  frustration  scores  by  display  option. 


NASA  TLX  Frustration 

Mean 

Std.  Deviation 

N 

DISPLAY  A 

8.98 

9.23 

32 

DISPLAY  B 

5.89 

7.14 

32 

DISPLAY  C 

3.92 

5.11 

32 

DISPLAY  D 

10.21 

9.58 

32 
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Table  32.  Follow-on  paired  comparison  analyses  for  mean  NASA  TLX  weighted 
frustration  scores. 


Paired  Comparisons 

t 

N 

Sig.  (2-tailed) 

Pair  1 

A  versus  B 

1.774 

31 

.086 

Pair  2 

A  versus  C 

3.413 

31 

.002 

Pair  3 

A  versus  D 

-.624 

31 

.537 

Pair  4 

B  versus  C 

1.400 

31 

.171 

Pair  5 

B  versus  D 

-2.911 

31 

.007 

Pair  6 

C  versus  D 

-3.529 

31 

.001 

3.3.8  Soldier  Overall  Preference 

Soldiers  were  requested  to  rank  order  the  displays  in  the  order  of  their  preference,  with  1  being 
their  favorite,  2  their  second  choice,  etc.  Table  33  displays  the  results  from  this  ranking. 


Table  33.  Mean  subjective  rankings  for 
display  preferences. 


MEAN 

A 

B 

C 

D 

2.66 

1.93 

2.23 

3.21 

Soldiers  ranked  the  displays  (table  33)  in  the  same  order  that  they  rated  the  displays  for  overall 
ability  to  perform  driving  tasks  (table  18).  Once  again,  they  demonstrated  that  they  preferred  the 
two  smaller  displays  to  the  largest  display  and  GMD.  Figure  13  displays  the  number  of  times 
that  each  display  size  was  chosen  first,  second,  third,  and  fourth. 


Figure  13.  Number  of  times  each  display  was  chosen  in  the  first  two  positions  and  the  last  two  positions. 
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4.  Discussion 


It  is  important  to  consider  the  tasks  that  were  examined  during  this  field  experiment  when  one  is 
interpreting  the  results.  The  tele-operation  task  was  confined  to  a  marked  course  (Legs  A  and  C) 
and  to  a  very  short  off-road  movement  around  an  obstacle.  The  target  detection  task  was  limited 
to  targets  within  the  immediate  vicinity  of  the  robot.  The  “bottom-line”  finding  from  this  experi¬ 
ment  is  that  Soldiers  were  able  to  perform  the  tele-operation  and  detection  tasks  required  by  this 
course  with  all  four  devices.  All  Soldiers  completed  the  courses  in  similar  amounts  of  time,  with 
similar  numbers  of  errors.  Additional  examination  of  the  results,  the  literature,  and  the  Soldier 
comments  sheds  further  light  on  this  finding. 

Although  the  Soldiers  performed  significantly  worse  with  device  D  (GMD)  on  only  one  measure 
(Leg  B  completion  times),  please  note  that  there  was  a  trend  for  worse  perfonnance  with  device 
D  on  every  performance  measure.  Soldiers  found  it  to  cause  significantly  more  work,  effort,  and 
frustration  than  Displays  B  and  C  and  more  mental  demand  than  all  three  of  the  other  displays. 

It  was  also  the  least  preferred  device  and  caused  greatly  increased  physical  problems.  Soldiers 
complained  of  too  much  visual  stimulation  because  of  the  strong  visual  images  presented  in  each 
eye  and  binocular  rivalry.  Binocular  rivalry  is  a  term  used  when  dissimilar  images  are  presented 
to  the  two  eyes,  which  causes  an  unstable  state  of  alternating  periods  of  monocular  dominance. 
Overall  dominance  often  appears  as  a  fragmented  mixture  of  the  two  eyes’  views  resulting  in  a 
dynamic,  patchwork  appearance  (Lee  &  Blake,  1999;  Mazumder,  Drury,  &  Helander,  1997; 
Meenes,  1999).  Most  of  the  Soldiers  closed  their  left  eye  or  covered  it  with  their  hand  in  an 
attempt  to  reduce  the  problem. 

On  the  positive  side,  the  GMD  (device  D)  used  during  this  experiment  was  reported  to  be  comfort¬ 
able  to  wear  (lightweight).  Soldiers  did  express  concern  that  the  design  did  not  let  them  wear  their 
ballistic  eye  protection,  but  the  manufacturer  stated  that  there  were  plans  to  accommodate  this 
problem.  There  were  no  reports  of  fogging  (as  is  frequently  the  case  with  night  vision  goggles 
and  some  closed  goggles  such  as  the  sun,  wind,  and  dust  goggles). 

The  smaller  display  sizes  (devices  B  and  C)  did  not  appear  to  cause  any  problems  for  these 
Soldiers  during  the  time  periods  when  they  used  them.  Most  of  the  Soldiers  positioned  themselves 
closer  to  the  smaller  screens  than  they  did  to  device  A,  which  may  have  caused  more  accommoda¬ 
tion  and  convergence  of  their  eyes.  Both  accommodation  and  convergence  have  been  cited  as 
contributing  to  eye  strain  (Jaschinski-Kruza,  1988;  Owens  &  Wolf-Kelly,  1987).  However,  the 
accommodative  and  convergent  demand  required  when  the  smaller  screen  sizes  were  moved  closer 
to  the  Soldiers’  eyes  to  achieve  better  quality  of  inherent  magnification  did  not  appear  to  cause 
discomfort  for  the  periods  of  time  they  used  the  displays.  In  fact,  the  Soldiers  rated  the  comfort  of 
viewing  devices  B  and  C  higher  than  they  did  the  comfort  of  viewing  the  larger  display  (device  A) 
and  the  GMD  (device  D).  This  may  have  been  because  of  their  fairly  young  ages  (most  of  the 
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Soldiers  were  under  30),  or  the  viewing  distance  required  by  the  smallest  display  size  was  not 
close  enough  to  cause  problems.  This  hypothesis  is  supported  by  a  study  of  three  viewing 
distances  (24,  33,  and  39  inches)  in  which  Rempel,  Willms,  Anshel,  Jaschinski,  and  Sheddy  (2007) 
found  that  the  near  distance  was  associated  with  significantly  less  blurred  vision,  less  eye  irritation, 
fewer  headaches,  and  improved  convergence  recovery  than  the  middle  and  far  distances. 

The  findings  that  Soldier  performance  with  the  smaller  display  sizes  (devices  B  and  C)  was  equal 
to  perfonnance  with  the  larger  display  size  (device  A)  are  consistent  with  several  other  experi¬ 
ments  that  found  no  performance  differences  created  by  the  effect  of  display  size  (Alexander, 
Wickens,  &  Hardy,  2005;  Stelzer  &  Wickens,  2006;  Stark,  Comstock,  Prinzel,  Burdette,  &  Scerbo, 
2001;  Muthard  &  Wickens,  2004). 

Although  the  Soldiers  did  not  perform  significantly  worse  with  the  largest  display  (device  A)  than 
they  did  with  the  smaller  displays  (devices  B  and  C),  there  was  a  trend  for  worse  perfonnance  with 
device  A  than  with  devices  B  and  C  and  Soldiers  rated  it  significantly  more  frustrating  than 
Display  C.  Alexander,  Wickens,  and  Hardy  (2005)  hypothesized  that  a  smaller  display  could  be  of 
benefit  to  traffic  surveillance  by  pilots,  given  that  fewer  eye  movements  would  be  required  to  scan 
the  small  display  area  as  compared  with  a  larger  display  area.  One  Soldier’s  theory  for  why  he  did 
not  perform  as  well  with  Device  A  as  he  did  with  the  smaller  devices  was  consistent  with  this 
hypothesis.  However,  Martin-Emerson  and  Wickens  (1997)  found  little  added  cost  to  increasingly 
longer  eye  movements.  Another  similar  hypothesis  is  that  the  smaller  screens  present  infonnation 
within  the  subjects’  macular  vision,  which  is  responsible  for  the  clearest  and  most  distinct  vision 
and  limit  the  need  for  the  use  of  peripheral  vision  and  the  edges  of  the  screen.  Visual  input  to  the 
macula  occupies  a  substantial  portion  of  the  brain’s  visual  capacity,  and  the  smaller  display  sizes 
confine  the  information  from  the  robotic  scene  to  this  area  of  highly  concentrated  cones.  Further 
support  for  this  hypothesis  is  provided  by  Parasuraman  (1986)  who  reported  an  “edge  effect” 
which  he  described  as  an  observer’s  penchant  to  limit  monitoring  the  edge  of  displays.  Wickens, 
Muthard,  Alexander,  van  Olffen,  and  Podczerwinski  (2003)  found  that  this  edge  effect  resulted  in 
peripheral  changes  being  less  likely  to  be  detected  and  the  possibility  of  additional  effort  and  neck 
strain.  The  larger  displays  could  thus  hinder  SA  because  a  larger  display  area  would  fall  into  the 
peripheral  vision  of  the  operator.  However,  two  experiments  conducted  by  Muthard  and  Wickens 
(2004)  to  assess  the  impact  of  display  size  on  flight  control,  air  space  surveillance,  and  goal- 
directed  target  search  showed  mixed  support  for  these  hypotheses.  Pilots  exhibited  less  flight  path 
error  with  larger  displays,  and  surveillance  and  search  were  unaffected  by  display  size. 

A  factor  that  probably  influenced  the  Soldiers’  ratings  of  the  largest  display  was  the  mismatch 
between  the  camera  and  the  display  resolutions,  and  this  could  have  even  had  some  effect  on  the 
objective  results.  They  complained  that  the  system  did  not  have  good  resolution  although  it  had 
the  highest  resolution  of  the  devices  used.  The  resolution  of  the  10.4-inch  screen  (Device  A)  was 
800x600,  but  the  source  resolution  of  the  camera  was  only  640x480.  The  match  between  the 
camera  and  display  resolution  with  Device  B  could  be  the  reason  they  rated  the  middle  size 
screen  as  having  the  sharpest  image.  However,  the  camera  used  during  this  field  experiment  has 
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a  resolution  that  is  typical  of  cameras  on  small  robots  so  the  resulting  mismatch  could  easily 
occur  if  larger  displays  with  higher  resolution  are  used. 

As  noted  in  the  results  section,  several  problems  experienced  during  this  field  experiment  were 
typical  of  robotic  field  operations  and  had  nothing  to  do  with  display  size.  In  field  operations, 
operator  performance  is  sometimes  degraded  because  of  problems  with  robotic  systems.  Carlson, 
Murphy,  and  Nelson  (2004)  reviewed  data  from  10  studies  of  15  different  UGVs  and  found  that 
reliability  of  UGV  performance  in  the  field  tends  to  be  low  (i.e.,  between  6  and  20  hours  between 
failures).  The  common  causes  included  limited  wireless  communication  range  and  insufficient 
bandwidth  for  video-based  feedback.  These  issues  also  often  adversely  impacted  the  human 
operator’s  remote  perception  and  ability  to  tele-operate.  Two  problems  with  connectivity  and 
communication  were  especially  prevalent.  The  first  one  was  the  frequent  lag  in  video  reception, 
which  caused  operators  to  have  delayed  responses.  The  other  problem  was  similar.  Several  times, 
operators’  video  pictures  froze  and  the  operators  thought  that  the  system  was  not  responding. 

Thus,  they  continued  to  repeat  the  same  command.  In  realty,  the  system  was  responding  but  the 
operator  received  no  feedback,  so  the  continued  commands  resulted  in  the  operator  driving  the 
vehicle  too  far  forward  or  backward. 

Another  problem  that  was  noted  during  the  experiment  is  glare.  During  this  experiment,  the 
Soldiers  worked  from  a  tent  that  helped  reduce  the  problem  with  glare.  However,  dismounted 
Soldiers  will  not  often  have  the  luxury  of  working  from  a  tent  so  that  they  are  shaded  from  the 
sun,  and  glare  is  a  definite  problem  that  needs  to  be  addressed. 


5.  Conclusions  and  Recommendations 


Smaller  devices  can  be  used  to  display  video  that  is  sufficient  for  tele-operation  and  local  surveil¬ 
lance  with  small,  slow-speed  robots.  This  makes  more  screen  space  available  for  other  objects  (if 
the  total  display  size  is  larger  than  3.5  inches)  and  enables  the  use  of  lower  resolution  cameras 
which  would  allow  faster  video  processing  since  there  are  fewer  pixels  to  be  processed  and  less 
bandwidth  is  required  if  fewer  pixels  need  to  be  transmitted.  This  finding  does  not  address  map 
display  or  touch  screen  operation  and  is  limited  to  operators  tele-operating  robots  while  they  are 
seated  under  a  tent.  Visual  attendance  should  not  be  wider  than  the  5-degree  macular  vision  (the 
actual  width  depends  upon  the  viewing  distance)  in  order  to  avoid  visual  scanning  which  can  result 
in  fatigue  and  missed  data.  Monocular  helmet-  or  head-mounted  displays  should  be  carefully 
assessed  to  ensure  that  binocular  rivalry  and  over-stimulation  does  not  occur. 
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Other  factors  of  scalability  that  should  be  addressed  in  subsequent  experiments  include 

•  the  type  of  control  device  being  used  (i.e.,  mouse,  touch  screen,  keyboard,  etc.); 

•  structure  or  organization  of  the  content  of  an  interface  (a  desktop  computer  may  use  a 
presentation  that  is  optimized  when  a  high  resolution  monitor  is  used,  while  a  user  of  a  PDA 
might  view  the  infonnation  in  a  text-only  presentation); 

•  tactics,  techniques,  and  procedures  of  dismounted  robotic  control,  multimodal 
displays,  and  information  requirements  of  users  in  different  environments. 

Further  research  on  factors  that  may  have  an  adverse  impact  on  helmet-  or  goggle-mounted 
robotic  displays  needs  to  be  performed  if  plans  are  made  to  use  these  displays  for  robotic  tele¬ 
operation.  Such  factors  include  approaches  to  reduce  interference  between  the  scenes  presented 
to  two  different  eyes,  potential  problems  with  prolonged  occlusion  of  one  eye,  the  inability  to  use 
compensatory  head  movements  to  center  the  display  on  the  visual  field,  and  the  effects  of  eye 
dominance  on  the  use  of  monocular  displays. 
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Appendix  A.  Soldier  Demographics 


DEMOGRAPHICS 
SAMPLE  SIZE  =  32 

RANK  MOS/DUTY  POSITION  AGE 

E-4  -  1  E-6  -  7  Officer  Candidate  3 1  years  (mean) 

E-5  -14  E-7-8 

1 .  How  long  have  you  served  in  the  military?  8  years,  3  months  (mean) 

2.  a.  Have  you  had  an  infantry-related  job?  12  yes  20  no 
b.  If  yes,  how  long?  5  years,  7  months  (mean) 

3.  a.  Have  you  been  a  fire  team  leader?  9  yes  23  no 
b.  If  yes,  how  long?  1  year,  9  months  (mean) 

4.  a.  Have  you  been  a  squad  leader?  12  yes  20  no 
b.  If  yes,  how  long?  3  years,  8  months  (mean) 

5.  a.  Have  you  been  deployed  overseas?  22  yes  10  no 
b.  If  yes,  how  long?  2  years,  2  months  (mean) 

6.  a.  Have  you  been  deployed  in  a  combat  area?  17  yes  J_5  no 
b.  If  yes,  how  long?  1  year  (mean) 

7.  Corrected  visual  acuity:  27  20/20  both  eyes  J_  20/20  in  one  eye 

_J_  20/20  and  20/40  J_  20/25  and  20/25 

jlnr 

8.  Please  list  any  visual  problems  you  may  have.  None  listed. 

9.  What  is  your  height?  69  (inches)  (range  is  60-80) 

10.  What  is  your  weight?  172  (pounds)  (range  is  130-260) 

1 1 .  Which  hand  do  you  most  often  write?  28  right  _3_  left  J_  NR 

12.  Which  hand  do  you  most  often  fire  a  weapon?  28  right  _3_  left  J_  NR 
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13. a.  Do  you  wear  prescription  lenses? 


7_  yes  25  no 


b.  If  so,  which?  _3_  glasses  _4_  contact  lenses 

14.  Do  you  wear  glasses  or  contacts  while  firing  a  weapon?  24_  yes  _6_  no  _2_  NR 

15.  Which  is  your  dominant  eye?  22  right  8  left  2  NR 

16.  Have  you  every  driven  a  robotic  vehicle?  J_  yes  31  no 
EOD  robots  (1) 

17.  Have  you  ever  driven  a  remote  control  car?  27  yes  _5_  no 

18.  Using  the  scale  below,  rate  your  level  of  experience  with  the  following  computer  software 
and  computer  related  activities: 


1 

2 

3 

4 

5 

6 

No 

Below 

Slightly 

Average 

Slightly  above 

Above 

experience 

average 

below 

average 

average 

average 

MEAN  RESPONSE 

a.  Microsoft  Windows  98,  2000,  XP,  etc. 

4.50 

b.  Computer  based  games 

3.71 

c.  Army  digital  systems  (e.g.  FBCB2) 

2.23 

d.  I  would  self  rate  my  computer  skills  as: 

4.47 

19.  Using  the  scale  below,  please  rate  the  following  Knowledge,  Skills,  and  Abilities  (KSA) 
related  to  infantry  duties: 


1 

2 

3 

4 

5 

6 

No 

Below 

Slightly 

Average 

Slightly  above 

Above 

experience 

average 

below 

average 

average 

average 

MEAN  RESPONSE 

a.  Knowledge  of  tactics,  techniques,  and  procedures  (TTP). 

4.00 

b.  Knowledge  of  map  reading  and  orientation  in  field  setting. 

4.84 

c.  Knowledge  of  land  navigation. 

5.16 

d.  Knowledge  of  reconnaissance,  surveillance,  and  target  acquisition  procedures. 

3.34 

e.  Knowledge  relating  to  communications  equipment  and  communications  procedures. 

3.75 

f.  Communication  skills  (ability  to  use  communications  equipment  &  face-to-face 
communications  to  enhance  mission  accomplishment). 

4.63 
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TRAINING 
SAMPLE  SIZE  =  32 


1 .  Using  the  scale  below,  please  rate  the  training  that  you  received  in  the  following  areas: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN 

a.  flow  to  drive  the  robot 

6.31 

b.  Time  provided  to  practice  driving  the  robot 

6.06 

c.  flow  to  complete  Leg  A  (driving)  of  the  course 

6.50 

d.  flow  to  complete  Leg  B  (obstacle  negotiation)  of  the  course 

6.31 

e.  flow  to  complete  Leg  C  (target  detection  and  identification) 
of  the  course 

6.41 

f.  Evaluation  of  the  practice  lane 

6.16 

g.  How  to  identify  different  types  of  IEDs  found  on  the  course 

6.19 

h.  How  well  do  you  expect  to  perform  on  the  actual  course 

5.72 

i.  Overall  evaluation  of  the  training  course 

6.25 

2.  What  were  the  easiest  and  hardest  training  tasks  to  learn? 


Comments  No.  of  Responses 


Easiest 

All.  2 

Driving  the  Talon.  12 

Controlling  the  robot.  2 

Speed  and  direction.  1 

Target/IED  identification.  9 

Understanding  and  negotiating  the  course.  2 

Forward-rear  mount.  1 

Leg  C.  1 

Driving  between  the  lanes.  2 

Hardest 

Depth  perception.  1 

Getting  accustomed  to  how  the  robot  moves.  1 

Maneuvering  the  robot  effectively,  and  at  a  faster  pace.  2 

Driving  the  robot.  2 

Negotiating  obstacle  at  end  of  lane.  1 

Staying  in  the  lane.  1 
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Comments 


No.  of  Responses 


Turning  sensitivity  of  robot.  1 

Controlling  the  robot  (joystick).  1 

L-R  sensitivity  of  joystick.  1 

Adjusting  to  the  use  of  one  eye.  2 

Target  distribution  on  the  monitor.  1 

Identifying  the  objects  on  the  screen.  1 

Identifying  smaller  items.  1 

Identifying  some  things  because  of  shadows.  2 

The  sun  made  it  hard  to  see  and  identify  sometimes.  6 

Overcoming  negative  visual  limitations  (seeing  through  the  camera  with  sunny  spots  1 
or  not  being  able  to  look  around  without  moving). 

Target  identification.  8 

Leg  A.  1 

Viewing  on  the  training  screen.  1 

3 .  What  are  your  comments  on  the  training  course? 

Excellent  course.  4 

Good,  short  course.  1 

Good  training.  4 

Well  organized.  2 

Very  professional.  1 

Extremely  beneficial  to  the  execution  of  the  actual  course.  1 

Easy  and  complete.  1 

Easy  to  learn  and  use  gear.  1 

How  useful  these  robots  will  be.  1 

Passionate  staff  that  is  very  excited.  1 

Robot  is  easier  to  maneuver  than  I  expected.  1 

It  is  sufficient  enough  to  get  a  feel  for  operating/negotiating  the  Talon  along  the  1 

course. 

I  have  a  fair  amount  of  stick  time  (Iraq)  on  the  Talon.  It’s  good  to  see  improvements  1 
being  tested. 

Because  of  the  technical  difficulties  with  the  system  (picture),  I  was  unable  to  1 

maximize  its  use  on  the  lane. 

I  was  very  impressed  with  the  way  the  robot  handled.  The  camera  and  monitor 
combination  were  decent,  but  could  definitely  be  improved. 

More  training  on  identifying  suspicious  objects  needed.  1 

The  obstacles  weren’t  always  easy  to  see  or  identify.  1 

The  screen  gets  a  lot  of  distortion;  flickers  on  rough  spots  on  the  track.  1 

Could  use  more  time  get  the  look  of  the  screen  and  the  feel  of  the  controls.  1 

If  camera  was  not  as  jumpy,  would  be  helpful.  1 
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POST  ITERATION 
SAMPLE  SIZE  =  32 


1 .  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following  navigation 
tasks  based  on  your  experience  with  the  display  that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  Very  Difficult  Neutral  Easy  Very  easy  Extremely 

difficult  difficult  easy 


NAVIGATION  TASKS 

MEAN 

A 

B 

C 

D 

a.  Move  in  the  correct  direction 

5.63 

5.69 

6.00 

5.16 

b.  Avoid  obstacles 

5.50 

5.62 

5.63 

4.91 

c.  Avoid  pot  holes 

5.07 

5.28 

5.17 

4.50 

d.  Assess  down  slopes  for  navigability 

5.03 

5.21 

5.19 

4.43 

e.  Assess  side  slopes  for  navigability 

4.97 

5.14 

5.23 

4.43 

f.  Identify  any  other  terrain  features  that  might  have  an  adverse 
effect  on  the  ability  of  the  robot  to  maneuver  through  the  terrain 

5.06 

5.03 

5.35 

4.48 

g.  Localize  the  robotic  vehicle  within  the  environment  so  that 
you  can  tell  where  the  sides  of  the  vehicle  are  located 

5.41 

5.31 

5.66 

4.94 

h.  Localize  the  robotic  vehicle  within  the  environment  so  that 
you  can  tell  where  the  front  of  the  vehicle  is  located 

5.91 

5.72 

6.00 

5.34 

i.  Localize  the  robotic  vehicle  within  the  environment  so  that  you 
can  tell  where  the  back  of  the  vehicle  is  located 

4.65 

4.58 

4.84 

4.29 

j.  Anticipate  whether  the  ground  clearance  of  the  vehicle  will 
allow  negotiation  of  rugged  terrain 

4.75 

4.68 

4.87 

4.22 

k.  Anticipate  whether  the  turn  radius  of  the  vehicle  will  allow  a 
turn 

5.47 

5.59 

5.53 

5.06 

1.  Identify  whether  you  have  navigated  through  this  terrain  in  the 
past. 

4.90 

4.90 

5.20 

4.53 

m.  Identify  if  you  are  on  the  correct  path 

5.13 

5.38 

5.59 

4.91 

n.  See  far  enough  ahead  to  plan  route  in  advance 

5.16 

5.25 

5.38 

4.81 

o.  See  well  enough  to  drive  at  slowest  speeds 

5.47 

5.78 

5.94 

5.41 

p.  See  well  enough  to  drive  at  medium  speeds 

5.03 

5.22 

5.53 

4.69 

q.  See  well  enough  to  drive  at  fastest  speeds 

4.28 

4.63 

4.88 

3.94 

r.  Overall  ability  to  perform  driving  tasks 

5.03 

5.31 

5.63 

4.69 
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Comments 


No.  of  Responses 


Display  A 

Great  size,  view.  1 

Overall  driving  ability  was  good.  1 

Very  easy  to  navigate.  2 

Easy  to  navigate  with  big  screen.  1 

Steady  picture  throughout.  1 

Speed  and  turn  ability  of  the  vehicle  is  fine.  1 

Much  easier  to  see  all  targets.  1 

Robot  was  easy  to  move,  but  more  difficult  to  turn  and  keep  on  track  at  the  same  1 
time. 

Navigation  was  off,  but  I  feel  that  had  a  lot  to  do  with  the  joystick  being  sensitive  1 
and  not  so  much  the  robot. 

For  the  most  part,  very  fluid.  1 

Video  quality  was  poor  throughout  lane.  1 

Resolution  was  very  bad.  1 

Reception  was  poor.  2 

Camera  distorting  view.  1 

Screen  picture  is  distorted  most  of  the  time.  Makes  it  difficult  to  identify  targets  and  3 
the  correct  path  to  travel. 

Difficult  to  navigate  over.  Robot  had  to  be  moved  manually.  2 

When  distorted,  pixels  were  way  too  big.  1 

Pixilation  main  issue  on  control  screen  which  hampered  my  control.  Had  to  stop  1 
several  times  to  wait  for  a  clearer  screen. 

Hard  to  judge  right  side  due  to  camera  placement.  1 

A  lot  of  technical  interference.  1 

Display  B 

Clearest  to  this  point.  3 

Very  easy  to  navigate.  3 

Picture  quality  high  except  for  a  few  spots.  1 

Great  picture  when  stationary,  but  deteriorates  rapidly  as  you  start  to  move.  1 

Camera  was  at  times  a  hindrance.  1 

Joystick  was  overly  sensitive,  which  made  it  hard  to  navigate.  1 

Improve  clarity  of  screen  resolution.  1 

Screen  was  blurred  and  distorted  at  times.  3 

Lots  of  picture  breakup  throughout.  I  had  limited  visibility  30%  of  the  route  and  1 
complete  blackout/whiteout  5%  of  the  time. 

PDA  size  was  better.  1 

Has  to  magnify  the  mega  pixels  too  much.  1 

The  wind  made  reception  very  poor.  1 

Bad  reception  to  view  camera  feed  from  robot.  1 

Display  C 

Excellent  view  of  area  to  navigate.  1 
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Comments 


No.  of  Responses 


Very  easy  to  navigate  even  with  smaller  screen.  2 

Worked  well.  1 

Performs  well  on  smooth  terrain  with  little  to  no  vegetation.  1 

By  far  best  screen  to  complete  tasks,  but  had  to  sit  and  pay  close  attention  to  screen  1 
due  to  size. 

Smaller  screen  allows  for  more  focus  while  navigating.  1 

Navigation  was  more  accurate  the  second  time.  1 

Picture  clarity  allowed  user  to  see  far  enough  to  adjust  for  proper  maneuver.  1 

Way  better  than  GMD,  and  no  lag  in  video.  1 

Other  than  flickering  screen,  I  could  see  fairly  well  to  navigate.  1 

Reception  was  very  bad  at  most  times.  1 

Poor  visibility.  1 

Poor  picture  quality.  1 

Difficulties  due  to  glare  and  broken  feedback.  Had  to  wait  a  few  seconds  for  good  1 
feedback  to  return. 

A  little  trouble  seeing  distance.  1 

L  to  R  axis  did  not  respond  well.  The  robot  to  controller  log  was  bad  at  times  and  I  1 
frequently  had  to  stop  while  attempting  L  to  R  movements.  We  even  had  to  turn 
off  the  control  consoled  once  to  try  to  fix  the  issue. 

Display  D 

Excellent  handling.  1 

Easier  than  I  thought  it  would  be.  1 

Control  seems  to  be  easier  with  this  monitor.  1 

The  turning  radius  and  full  view  picture  is  quite  accurate.  2 

Navigation  is  easy  except  during  faded  signals  and  screen  saturation  from  direct  1 

sunlight. 

Delay  on  the  monitor.  1 

Monitor  inoperability  (picture  in/out).  1 

Unable  to  identify  lane,  and  had  problems  trying  to  ID  and  maintain  situational  1 

awareness. 

Very  blurry  picture  with  a  lot  of  interference.  2 

Picture  was  often  broken  up  or  distorted  and  pixilated,  which  caused  driving  to  be 
more  difficult. 

Very  difficult  to  utilize  compared  to  the  screen  views.  1 

The  eyes  were  fitted  for  a  right-eye  dominant  wearer  and  I’m  left-eye  dominant,  so  2 
the  task  was  more  difficult. 

I  had  to  stop  the  vehicle  a  lot  in  order  to  navigate.  1 

Control  of  robot  difficult  due  to  grainy,  pixilated  view  in  glasses.  1 

Often  hard  to  see;  pixels  were  not  sharp.  2 
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Comments 


No.  of  Responses 


Not  sure  if  it  was  lighting,  but  there  were  sections  where  the  course  was  not  visible.  1 
It  was  hard  to  see  in  front,  much  less  to  the  side.  1 

Could  not  see  distinctly  to  navigate  well.  Extremely  frustrating.  1 

Kept  losing  camera.  1 

Static  and  sticking  when  bumpy.  1 


2.  Using  the  scale  below,  please  rate  your  ability  to  ability  to  maintain  situation  awareness 
around  the  vehicle  with  the  display  that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  Very  difficult  Difficult  Neutral  Easy  Very  Extremely 

difficult  easy  easy 


SITUATION  AWARENESS  TASKS 

MEAN 

A 

B 

C 

D 

a.  Locate  mines 

3.97 

4.73 

4.50 

3.94 

b.  Locate  IEDs 

4.31 

4.84 

4.78 

4.19 

c.  Locate  booby  traps 

4.00 

4.57 

4.43 

3.82 

d.  Locate  enemy  personnel  targets 

5.84 

5.94 

5.94 

5.31 

e.  Overall  situation  awareness  of  the  environment  around  the 
vehicle 

4.75 

5.16 

5.28 

4.66 

Display  A 

Good  platform;  easier  to  estimate  size  and  distance.  1 

Could  see  better  with  this  screen,  but  peripheral  vision  was  limited  so  couldn’t  see  1 
until  past  the  object. 

With  exception  of  resolution/contrast  issues  due  to  weak  signal,  overall  situation  1 
awareness  is  mostly  improved  over  ah  other  screen  sizes. 

Most  of  the  targets  were  easily  identifiable.  The  personnel  targets  were  the  easiest;  1 
mines  were  the  most  difficult. 

Better  to  ID  smaller  IEDs,  than  other  instruments.  1 

Very  difficult  to  identify  smaller  objects.  1 

Hard  to  detennine  IED  vs.  natural  objects  at  times.  1 

Display  had  a  lot  of  interference.  1 

Camera  going  in  and  out.  1 

Screen  picture  is  distorted  which  makes  it  difficult  to  distinguish  a  tree  from  a  1 

personal  target.  Pipe  and  pressure  bombs  are  definitely  impossible  to  identify  on 
the  given  terrain. 

Picture  clarity  affected  situation  awareness.  2 
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Comments 


No.  of  Responses 


If  there  was  a  way  to  minimize  visual  distortion,  it  would  be  very  easy  to  locate  1 

targets. 

Resolution  should  be  improved.  1 

Pixilation  of  camera  view  was  severe.  1 

The  rougher  the  terrain,  the  more  pixilation.  Harder  to  see  while  moving  at  full  1 

speed. 

Difficult  to  find  engineer  tape  on  the  ground  after  negotiating  obstacle.  1 

Lost  picture  occasionally.  1 

Display  B 

Very  easy  to  see  enemy  personnel  and  mines.  1 

Picture  allowed  for  situation  awareness.  1 

Pretty  easy  to  differentiate  size  and  natural  vs.  man-made.  1 

Display  not  big  enough  at  times.  1 

I  received  a  weak  signal  throughout  the  majority  of  the  exercise  resulting  in  poor  1 

image  quality. 

Resolution  needs  improving.  1 

Had  difficulty  when  screen  was  blurry.  1 

Had  to  stop  periodically  to  identify  any  potential  targets.  1 

Clarity.  1 

When  moving,  image  was  choppy  and  would  freeze.  2 

Would  be  better  if  there  was  no  screen  breakup.  1 

Camera  made  it  very  difficult  to  identify  targets.  A  lot  of  fuzziness  and  camera  1 

going  in  and  out. 

Display  C 

Easy  to  ID  personnel  targets.  1 

Better  video  and  little  lag  time.  1 

Still  very  easy  to  locate  IEDs  and  enemy  personnel.  2 

At  times  there  was  technical  interference  with  the  picture,  but  mostly  you  could  see  1 
what  was  going  on  around  you. 

Identification  depends  mainly  on  clarity  of  visual  pixilation.  1 

Location  was  easy,  but  differentiating  size  can  be  tough.  1 

Easier  to  use  my  peripheral  vision  than  on  the  glasses.  1 

If  reception  was  better,  it  would  be  easier.  1 

Picture  quality  and  sunlight  created  many  reflections,  which  were  at  times  mistaken  1 
as  IEDs. 

Screen  fade  outs,  and  sun  (direct  light)  made  some  areas  of  course  difficult  to  gain  1 
situational  awareness. 

Only  have  view  to  front  and  no  audio.  This  makes  it  difficult  to  have  situational  1 
awareness. 

Small  screen  made  it  tough  at  times.  1 

Hard  to  ID  IEDs  because  of  scale.  1 

It  was  very  difficult  to  see  at  times  because  the  camera  was  very  scratchy.  1 


37 


Comments 


No.  of  Responses 


Display  D 

Display  is  very  good.  1 

Observation  to  the  front  is  ideal.  1 

Takes  time  to  become  adjusted  to  picture  with  one  eye.  Puts  strain  on  it  and  could  4 
become  tiring  after  awhile. 

Focus  on  the  eyepiece  was  not  100%.  1 

Observation  from  the  sides  is  limited  and  there  are  no  rear  views  available.  1 

Faded  signal  screen  saturations  are  major  factors  in  situational  awareness.  1 

Often  the  screen  was  unclear,  static  or  distorted  so  it  made  finding  smaller  objects  4 
unclear. 

Targets  appeared  in  the  screen,  but  without  prior  knowledge  of  what  they  look  like,  1 
it  would  be  very  difficult  to  ID  them.  It  was  not  easy  to  ID  something  out  of  place 
in  the  lane  because  of  the  picture  display. 

Camera  visibility  made  it  difficult  to  identify  targets  at  distance.  1 

Very  difficult  to  ID  targets.  3 

The  only  obvious  targets  were  the  personnel  targets.  2 

Difficult  to  determine  size  and  differential  between  natural  objects  and  IEDs.  1 

A  lot  of  interference  with  this  route.  I  had  to  stop  several  times  for  the  picture  to  2 
clear  up  enough  for  me  to  continue. 


3.  During  this  trial,  did  you  experience  any  of  the  following? 


Number  ol 

F  Responses 

A 

B 

C 

D 

Eyestrain 

6 

4 

2 

21 

Tunnel  vision 

1 

1 

1 

2 

Headaches 

0 

0 

0 

1 

Motion  sickness 

0 

0 

0 

3 

Nausea 

0 

0 

0 

1 

Disorientation 

1 

1 

0 

4 

Dizziness 

0 

0 

0 

1 

Any  other  problems? 

1 

0 

0 

4 

Display  A 

Size  of  screen  was  adequate.  1 

Not  as  much  static  with  this  screen  and  closer  to  the  ground  so  could  see  terrain  1 

better. 

Choppy  screen.  1 
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Comments 


No.  of  Responses 


Video  in/out;  required  constant  squinting.  1 

Video  quality  poor;  made  task  extremely  difficult.  4 

Hard  to  distinguish  objects.  1 

Feedback  too  broken  to  visualize.  1 

Peripheral  vision  limited.  1 

Disorientation  due  to  static  on  screen.  1 

Display  B 

Good  size  screen.  1 

Eyestrain  when  trying  to  identify  targets  at  times.  1 

Eyestrain  caused  by  fuzzy  screen.  1 

Screen  blurred  and  distorted  at  times.  1 

Glare  on  the  course  and  at  the  monitor  in  early  morning.  1 

Screen  was  jumpy;  hard  to  focus.  1 

Disorientation  due  to  sun  reflection.  1 

Display  C 

Size  of  picture  was  average.  1 

PDA  size  of  monitor  works  much  better  than  the  GMD  system  due  to  its  size.  1 

Targets  are  easier  to  identify  without  having  to  strain  your  eyes.  1 

Eyestrain  because  camera  fuzzy.  1 

Eyestrain  and  tunnel  vision  due  to  screen  size.  1 

Could  foresee  headaches/migraines  if  watched  through  this  screen  a  lot.  1 

Display  D 

Overall,  good  system.  1 

Screen  perhaps  can  be  a  little  bigger.  2 

Screen  fuzzy  at  times.  1 

Glasses  strained  my  vision.  Combination  of  bad  screen  and  only  one  screen  for  one  1 
eye. 

Only  see  directly  in  front.  1 

I  wasn’t  relaxed  with  my  vision;  was  moving  my  head  around.  1 

Blocking  out  right  side  would  help.  1 

At  times  I  had  to  close  one  eye  in  order  to  see.  2 

Had  to  concentrate  on  keeping  non-HUD  eye  closed  and  focus  dominant  eye  on  1 

viewing  at  very  close  screen. 

Eyestrain  because  of  squinting.  2 

Eyestrain  due  to  trying  to  focus.  1 

Right  eye  fatigue.  7 

Had  to  close  both  eyes  to  get  the  best  visual.  1 

Would  have  been  easier  to  use  my  left  eye  since  I  am  left  eye  dominant.  2 

I  would  develop  a  headache  if  used  for  extended  period  of  time.  2 
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Comments 


No.  of  Responses 


Had  no  clue  what  I  was  looking  at.  1 

Disorientation  after  major  video  disruption.  1 

Disorientation  when  spun  around  too  quickly.  1 

Visual  distortion.  2 

Nausea  like  I  was  driving.  1 

Dizziness  from  no  focus.  1 


Motion  sickness  due  to  the  rapid  movement  to  focus  in  small  monitor.  1 

4.  Using  the  scale  below,  please  rate  the  following  characteristics  of  the  display  that  you  just  used: 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


DISPLAY  CHARACTERISTICS 

MEAN 

A 

B 

C 

D 

a.  Resolution  (clarity) 

3.47 

4.25 

4.75 

3.88 

b.  Size  of  objects  appearing  in  the  display 

5.19 

5.22 

4.81 

4.63 

c.  Comfort  of  viewing  the  display 

4.81 

5.12 

4.91 

4.00 

d.  Contrast  between  objects 

4.44 

4.91 

4.91 

4.29 

e.  Display  color 

4.50 

4.78 

4.84 

4.38 

f.  Display  brightness 

4.69 

4.84 

5.00 

4.47 

g.  The  effect  of  the  size  and  shape  of  the  display  on  mobility  of 
the  dismounted  Soldier 

4.81 

4.94 

5.09 

4.97 

h.  Adequacy  of  a  display  of  this  size  for  tele-operating  a  robotic 
vehicle 

5.31 

5.16 

4.97 

4.69 

5.  Using  the  scale  below,  what  is  your  overall  rating  of  the  display  that  you  used  this  iteration? 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


MEAN 

A 

B 

C 

D 

4.75 

5.06 

4.96 

4.40 

Display  A 

Good  size  display.  2 

Easy  to  use.  2 

No  difficulty  in  positioning  and  operation.  1 

Best  view  when  identifying  targets,  IEDs.  1 
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Comments 


No.  of  Responses 


When  screen  was  clear,  this  was  the  best  screen  to  operate  with.  1 

Screen  was  good  when  signal  was  good.  2 

I  like  the  PDA  view  when  driving,  and  the  big  screen  when  identifying  targets.  1 

Large  objects  can  be  easily  seen.  1 

Display  size  is  adequate.  3 

With  improvements  to  the  resolution,  the  display  would  be  great.  1 

This  screen  should  be  used  for  identifying  objects  close  to  the  ground  in  plain  view.  1 

This  size  screen  would  be  better  suited  for  a  vehicle.  It  may  be  too  bulky  for  1 

dismounted  soldier. 

Lots  of  interference,  but  otherwise  great.  1 

Get  a  better  resolution  camera  with  anti-shock  and  then  it  would  be  great.  1 

Technical  interference  of  display  was  only  problem.  1 

Video  quality/clarity  is  poor;  distorted.  8 

Visual  feedback  was  not  as  good  as  expected  for  the  size  of  the  display.  Feedback  1 
too  broken;  had  to  stop  many  times  during  mission  to  relocate  self  and  vehicle. 

Lack  of  clarity  made  it  very  difficult  to  distinguish  smaller  objects.  1 

Lack  of  contrast  and  almost  eye  blinding  brightness  at  places  didn’t  help.  1 

Had  difficulty  identifying  targets  with  the  morning  sun  coming  directly  into  the  2 

camera. 

Pixels  way  too  big.  1 

Pixilation  ruined  the  advantage  of  a  larger  screen.  Fix  this  issue  and  the  screen  size.  2 
There  was  less  pixilation  on  the  CDA  size.  I’d  go  with  CDA  only  because  of  that 
reason.  1 

Screen  kept  fading  out.  2 

Display  B 

Best  display  to  this  point.  4 

Display  was  very  good.  2 

Better  clarity  and  contrast.  3 

Good  blend  of  size  and  mobility  to  display  clarity.  1 

Better  driving  view  compared  to  the  big  screen.  1 

Could  navigate  quickly  and  distinguish  objects.  2 

The  display  was  small,  but  I  could  see  objects  of  interest.  1 

This  size  screen  would  be  adequate  given  the  resolution  was  better.  1 

View  was  good,  but  when  signal  was  weak,  unable  to  ID  or  see  smaller  devices.  1 
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Comments 


No.  of  Responses 


If  given  the  opportunity  to  periodically  stop  and  let  the  picture  focus,  the  system  1 

would  be  very  effective. 

Visual  distortion  while  moving  or  standing  was  high.  2 

Could  see  well  when  screen  wasn’t  scrambling.  1 

Screen  breakup  and  light  conditions  make  the  robot  much  harder  to  operate.  1 

Very  blurry.  1 

Camera  pixels  bad  on  rough  terrain.  1 

Resolution  unclear  at  times,  making  it  difficult  to  identify  objects  -  even  within  10  3 

meters. 

In  bright  sunlight,  some  objects  are  difficult  to  see.  1 

Still  too  bright.  1 

I  prefer  driving:  PDA  view.  1 

I  prefer  location  targets:  big  screen.  1 

Wind  really  presented  reception  problems.  1 

Would  not  recommend  this  display  for  the  size  of  the  robot.  Too  much  interference  1 
which  took  time  to  correct. 

Had  the  signal  been  strong  throughout  exercise,  I  believe  I  would  have  detected  a  1 
substantial  increase  in  man-made  objects.  On  several  targets,  I  could  not  get  a 
good  enough  image  to  discern  it  from  natural  and  man-made  objects. 

Display  C 

This  display  was  clearly  the  best  of  the  four.  1 

Best  screen  for  clarity.  1 

The  displays  were  mainly  good  throughout  test.  1 

Easy  to  navigate  course.  1 

I  would  like  to  work  with  this  system.  1 

Overall,  a  good  system.  2 

Visibility  and  clarity  were  extremely  good.  1 

It  was  better  driving  the  robot  compared  to  the  goggles.  Anticipation  on  where  I  1 
was  going  was  a  factor. 

For  the  dismounted  soldier,  I  feel  this  display  is  suitable  and  would  provide  ease  of  1 
storing  if  needed. 

When  receiving  correctly,  looks  very  adequate  to  accomplish  its  mission.  1 

Better  than  CDA,  but  not  quite  as  good  as  the  FBCB2.  1 

Display,  although  small,  could  be  used  very  easily.  1 

Would  serve  well  in  theater  with  some  enhancements.  1 

Clarity  was  good,  but  I  prefer  the  CDA  just  because  it’s  bigger.  1 

Clear,  but  the  brightness  was  too  high,  which  inhibited  distant  target  viewing.  1 

Sun  glare  made  it  difficult  to  see  in  certain  directions.  1 

Screen  too  small  for  me;  wouldn’t  like  to  look  at  this  long.  1 
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Comments 


No.  of  Responses 


Had  to  close  in  on  display  to  achieve  better  recognition.  1 

Display  too  small.  2 

Miss  some  level  of  detail  due  to  screen  size.  2 

Screen  kept  fading  out.  1 

Display  changed  colors  frequently  in  brighter  areas.  1 

The  reception  was  poor,  so  display  was  fdled  with  static,  and  lines  with  1 

discoloration.  I  needed  to  stop  multiple  times  to  allow  the  display  to  quit 
“freezing”  before  I  could  continue. 

A  lot  of  feedback  from  display  made  control  difficult  at  times.  1 

Signal  strength  needs  improvement  for  remote  use.  1 

Need  to  enhance  picture  quality  to  make  positive  ID  of  IED  and  any  other  perceived  1 
threat. 

There  was  a  delay  every  now  and  then  which  would  cause  me  to  have  to  stop  for  a  2 
minute  until  the  picture  caught  up. 

There  were  lots  of  flashes  and  lines  on  the  display  which  caused  it  to  be  difficult  to  1 
see  the  terrain,  objects,  and  lane. 

Camera  viewing  was  very  difficult  at  times.  1 

Display  D 

Overall,  good  equipment.  1 

I  was  really  impressed  with  the  clarity  of  the  picture  and  precision  of  the  turning  1 
radius. 

Focus  seemed  clearer.  1 

The  size  and  fit  would  be  ideal  for  a  dismounted  soldier.  1 

Adapted  to  size  quickly.  1 

Would  be  perfect  for  small,  light  units  if  picture  was  clearer.  1 

Although  awkward  getting  used  to  a  HUD,  it  provided  a  better  than  the  PDA  size  1 
viewer. 

Display  was  very  clear  while  the  robot  was  still,  but  broken  up  during  movement.  1 
If  the  eyepiece  (left/right  eye  dominant)  and  the  distortion  was  corrected,  this  would  2 
be  a  great  piece  of  equipment. 

Needs  great  improvement.  1 

Least  favorite  of  all.  1 

Better  definition,  but  still  way  too  bright  with  not  enough  contrast  between  objects.  1 
Could  be  interesting  to  try  under  different  light.  There  were  times  when  the  picture  1 
was  great  and  other  times  when  it  was  poor. 

A  little  weird  using  only  one  eye,  but  otherwise  the  resolution  and  color  were  good.  1 
If  there  could  be  a  way  to  see  the  display  with  the  right  eye  while  “blocking  out”  the  1 
left  eye,  it  would  allow  me  to  concentrate  a  lot  more  on  the  tasks  at  hand. 
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Comments 


No.  of  Responses 


Tried  to  drive  with  both  eyes,  but  it  was  easier  to  close  my  left  eye.  I  was  trying  to  1 
put  myself  in  a  real  situation  that  I  have  to  be  aware  of  my  surroundings  also  while 
my  other  eye  is  concentrating  on  the  display. 

This  display  method  is  extremely  uncomfortable  to  use  as  it  puts  strain  on  both  eyes.  1 
It  was  extremely  difficult  to  see  and  monitor  the  movements  of  the  robot  because 
of  poor  picture  quality  and  resolution. 

Uncomfortable  due  to  using  one  eye.  2 

Eye  fatigue  occurred  quickly  during  exercise.  1 

Display  sight  picture  was  at  times  glaring,  making  it  difficult  to  see  the  path  of  the  1 

objects. 

The  size  of  the  display  was  not  the  major  issue;  the  pixilation  and  poor  clarity  were.  1 
Everything  listed  in  “display  characteristics”  (i.e.,  picture,  brightness,  color  and  1 

objects)  needs  to  improve  to  make  this  system  a  more  effective  war  fighting  tool. 
Signal  strength  and  direct  sun  saturation  are  the  biggest  impediments.  1 

Eyepiece  lacks  clarity.  2 

Too  much  delay  in  video.  1 

This  was  the  most  difficult  viewing  thus  far.  1 

Not  soldier  friendly.  1 

Unable  to  visualize  or  figure  out  where  or  what  I  was  doing;  frustrating!  1 
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END  OF  EXPERIMENT 
SAMPLE  SIZE  =  32 


1 .  Please  rank  order  the  displays  in  the  order  of  your  preference  -  with  1  being  your  favorite,  2 
your  second  choice,  etc. 


MEAN 

A 

B 

C 

D 

2.66 

1.93 

2.23 

3.21 

Comments  No.  of  Responses 


Display  A 

Biggest  and  clearest.  2 

Easiest  to  pick  out  objects  on.  1 

Best  contrast  in  order  to  ID  potential  IEDs.  1 

Size  makes  it  rank  well  with  me.  3 

Good  tool.  1 

Easiest.  3 

Enjoyed  larger  screen,  but  pixilation  and  clarity  are  an  issue.  1 

Display  needs  to  be  improved.  1 

Largest  display  was  not  at  clear.  4 

Big  enough  for  target  ID,  but  when  reception  was  bad,  the  big  pixels  distorted  the  1 
picture. 

Broken  up  and  distorted.  4 

Resolution  is  bad.  1 

Couldn’t  keep  picture.  1 

Display  B 

Ranking  totally  based  on  maneuverability  and  ability  to  locate  IEDs,  etc.  I  would  1 
rather  have  the  best  equipment  than  the  equipment  that  is  the  lightest. 

Easy  to  use.  2 

Great  compromise  between  size  and  clarity.  3 

Good  size.  4 

Large  enough  to  be  comfortable  and  portable.  2 

Very  clear.  6 

OK;  some  static.  1 

Too  big  for  combat  soldier.  1 

Too  large  to  carry.  1 

Not  very  good.  1 

Bad  resolution  for  its  size.  1 
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Comments 


No.  of  Responses 


Display  C 

Good  clarity.  7 

Good  because  resolution  was  optimal  relative  to  the  size  of  the  display.  1 

Good  for  soldier  mobility.  1 

Best  mix  of  size  and  clarity.  1 

Easier  to  drive/navigate.  2 

Easy  to  use.  3 

No  problem  with  the  camera.  1 

Good  size;  easy  to  carry.  1 

Size  is  best  for  dismounted  troops.  1 

Most  adequate  size  relative  to  maneuverability.  1 

Tougher  than  the  others.  1 

Had  better  graphics.  1 

Liked  the  portability.  It  is  small  enough  and  can  be  easily  put  in  a  ruck  or  attached.  2 
Weighs  less  and  the  screen  size  is  optimal  for  a  good  view  to  ID  threat. 

Had  to  get  real  close  to  screen,  so  it  wasn’t  so  clear.  1 

Too  small.  3 

Last  resort.  Straining  eyes  an  issue.  1 

Display  D 

Size  is  best  for  dismounted  troops.  1 

I  like  the  size.  2 

Good  for  functionality.  1 

Ability  to  localize  objects  in  the  ground.  1 

The  most  convenient  item.  1 

Had  the  most  clarity.  1 

Good  idea,  but  needs  some  black  backdrop/shade  to  be  effective.  1 

The  clearest,  but  had  to  hold  goggles  close  to  face  in  order  to  use  well.  1 

I  felt  good  with  this  display  because  it  forced  me  to  pay  more  attention  because  I  1 
only  had  the  use  of  one  eye  and  I  thought  the  technical  side  was  the  best. 

Good  except  for  the  ability  to  use  only  one  eye.  2 

Could  be  very  good  if  clarity  was  increased.  1 

Made  my  eyes  tired  when  keeping  one  eye  closed  for  extended  period  of  time.  5 

Using  one  eye  gave  me  a  headache.  1 

The  eyes  cause  me  to  be  slightly  motion  sick.  1 

I  had  to  close  one  eye  the  entire  time  because  I  am  left-eye  dominant.  1 

While  still  clear  for  the  most  part,  it  was  somewhat  distracting  as  I  found  myself  1 
moving  my  head  in  a  futile  attempt  to  increase  the  view. 
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Comments 


No.  of  Responses 


Uncomfortable.  3 

Picture  was  way  too  small.  2 

Focus  was  not  good.  1 

Harder  to  use.  2 

Required  too  much  effort  to  see.  2 

Hard  to  see  through.  1 

The  resolution  (pixeling)  was  not  the  same  across  the  board.  1 

Most  ineffective  due  to  constraints  of  vision.  1 

2.  Do  you  have  any  suggestions  for  ways  to  increase  the  effectiveness  of  the  displays? 

The  displays  need  to  be  clearer;  less  distortion.  6 

Use  the  most  effective  display  that  produces  maximum  results  and  is  comfortable  to  1 
the  user. 

Improve  resolution.  3 

Reduce  the  fading  out  of  display.  1 

Work  on  pixilation.  2 

More  pixels  on  some  of  the  larger  displays.  The  feed  from  the  camera  gets  distorted  1 
quite  easily  with  movement. 

Possible  use  the  HUD  with  both  eyes.  1 

Increase  the  quality  of  viewing  picture.  Soldier  would  benefit  from  clear  picture,  1 
especially  concerning  IED  detection. 

Improve  overall  view  (color,  contrast,  brightness,  etc.).  2 

Adjust  the  brightness  so  that  clarity  can  be  maintained  in  bright  or  darker  conditions.  1 
Enhance  the  contrast  by  lowering  the  brightness.  The  glare  sometimes  diminished  1 
contrast  to  the  point  that  you  couldn’t  see  the  object  until  it  was  right  there. 

Match  the  view  screen  to  the  dominant  eye.  1 

Use  the  new  LCD  or  plasma  tech  with  system  to  enhance  its  display.  1 

Better  optics  and  increase  video  transfer  data  rate.  1 

If  the  camera  could  move  from  side  to  side  while  the  robot  is  stopped  to  check  on  1 
items  visualized. 

Cameras  to  monitor  displays  can  be  better.  1 

Better  visual  image  from  the  camera.  1 

The  ability  to  move  the  camera  while  leaving  the  robot  on  its  current  heading.  1 

Put  some  sort  of  sun  visor  on  the  camera.  Sun  made  it  difficult  to  see.  1 

Keep  the  weight  low  and  compact.  1 

Add  a  manual  zoom  adjustment.  1 
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Comments 


No.  of  Responses 


The  wind  affected  the  display  on  several  iterations.  2 

Shaking  of  display  over  bumps.  1 

Improve  signal  strength  to  improve  remote  operations.  2 

Better  signal  quality.  2 

I  prefer  the  FBCB2  size,  but  the  mega  pixels  seemed  stretched  and  more  grainy.  1 
Better  antennas  for  better  reception.  1 

The  transmission  and  reception  quality  needs  to  be  better  for  this  system  to  be  1 

combat  effective. 

Difficult  to  differentiate  between  shadows  from  the  trees  and  IEDs.  1 

3.  Do  you  have  any  suggestions  on  ways  to  improve  the  driving  training  course? 

No,  the  course  was  helpful.  6 

Good  training.  3 

White  engineer  tape  is  perfect  but  not  all  in  the  course.  1 

Use  a  different  color  engineer  tape  to  highlight  the  course.  Neon  colors  work  best.  1 

Have  more  than  one  practice  session.  1 

Should  be  trying  this  in  an  urban  area  and  roads.  Our  soldiers  are  not  operating  in  1 
the  woods  for  the  most  part  in  Iraq  and  Afghanistan. 

Integrate  more  urban  obstacles,  i.e.,  small  buildings,  cars,  etc.  to  better  simulate  the  1 
environment  soldiers  will  encounter  in  Iraq  or  Afghanistan. 

Give  trainees  a  course  where  they  can  see  their  robot  reacting  to  their  joystick  1 

movements. 

Course  was  set  up  well  with  different  kinds  of  terrain.  Maybe  a  little  more  sloping  3 
terrain. 

Incorporate  small  ditch  or  trench.  1 

Display  needs  to  be  improved.  1 

Go  from  smooth  surface  with  gentle  corners  to  off-roading  with  hills  and  complex  1 
corners. 

Point  out  what  each  item  is  on  the  screen  as  they  are  seen.  1 

More  turns  and  targets.  1 

Take  sun  into  effect  when  picking  route.  1 

Left/right  sensibility  a  little  fast.  1 

Have  a  smaller  hand  device.  1 

Improve  the  image  that  is  transmitted  to  image  screen  to  have  less  distortions.  1 
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4.  Do  you  have  any  suggestions  on  ways  to  improve  the  driving  lanes  that  you  negotiated? 


Comments  No.  of  Responses 

No,  good.  4 

Lanes  were  well  planned.  1 

It’s  a  good  thing  we  had  a  trial  first  before  we  went  to  the  lanes.  1 

Make  the  courses  a  little  more  difficult.  1 

Electronic  interference  was  the  only  bad  aspect  of  it.  1 

Incorporate  urban  terrain,  night  time  with  night  vision  and  desert-like  conditions.  2 

Try  to  use  training  aides  more  realistic  to  theater  (artillery  shells  or  mortars,  etc.).  1 

Include  slide  slopes,  potholes,  muddy  terrain,  and  water  holes  along  the  course.  1 

Have  buildings  and  curbs.  2 

More  hills.  These  are  harder  to  navigate  and  might  show  other  flaws.  4 

More  turns.  1 

Add  some  length  to  the  course;  more  targets.  1 

Don’t  stop  when  robot  goes  outside  of  lane.  Give  leeway  to  user  to  find  way  back.  1 
Take  sun  into  effect  as  well.  1 


5.  Using  the  scale  below,  please  rate  the  importance  of  the  following  information  that  is  needed 
for  tele-operating  a  robot. 

1  2  3  4  5  6  7 

Extremely  Very  Unimportant  Neutral  Important  Very  Extremely 

Unimportant  Unimportant  Important  Important 


MEAN 

a.  Heading  information 

5.16 

b.  Relative  distance  to  obstacles 

5.47 

c.  Depth  of  pot  holes 

4.81 

d.  Information  on  navigability  of  down  slopes  (tilt  meter) 

4.94 

e.  Information  on  navigability  of  side  slopes  (tilt  meter) 

5.03 

f.  Identification  of  any  other  terrain  features  that  might  have  an  adverse  effect  on 
the  ability  of  the  robot  to  maneuver  through  the  terrain 

5.41 

g.  Ability  to  determine  where  the  sides  of  the  vehicle  are  located,  (sides  are  in  the 
camera  field  of  view  or  are  marked  on  the  screen) 

5.31 
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(cont) 


MEAN 

h.  Ability  to  detennine  where  front  of  the  vehicle  is  located 

5.69 

i.  Ability  to  detennine  where  back  of  the  vehicle  is  located 

4.84 

j.  Infonnation  concerning  whether  the  ground  clearance  of  the  vehicle  will  allow 
negotiation  of  rugged  terrain 

5.12 

k.  Indication  of  the  turn  radius  of  the  vehicle 

5.16 

1.  Infonnation  on  the  tenain  closer  than  6  inches  in  front  of  the  vehicle 

4.94 

m.  Information  on  the  terrain  5  to  1 5  feet  in  front  of  vehicle 

5.66 

n.  Information  on  the  terrain  greater  than  15  feet  in  front  of  vehicle 

5.22 

o.  Information  on  current  speed  of  the  vehicle  (speedometer) 

4.34 

p.  Information  on  vehicle  revolutions  per  minute 

3.66 

q.  Information  on  the  way  the  vehicle  motor  sounds 

4.06 

r.  Infonnation  on  other  noises  present  in  the  vehicle’s  environment 

5.03 

s.  Information  concerning  the  color  of  objects  in  the  environment  (color  vs.  black 
and  white  camera) 

6.16 

t.  Infonnation  concerning  objects  on  the  side  of  the  vehicle  (side  facing  cameras) 

5.31 

u.  Information  concerning  objects  behind  the  vehicle  (back  facing  camera) 

5.28 

v.  Information  concerning  the  condition  of  the  vehicle  tires/tracks. 

4.44 

w.  Information  concerning  the  temperature  of  vehicle 

4.22 

x.  Feedback  on  the  ruggedness  of  terrain. 

4.81 

y.  Information  concerning  any  delay  between  the  time  the  operator  sends  a  vehicle 
command  and  the  time  that  the  vehicle  responds  (i.e.,  how  long  after  a  stop 
command  is  sent  before  the  vehicle  actually  stops) 

5.66 

z.  Feedback  concerning  whether  or  not  the  vehicle  correctly  received  a  command 

5.72 

aa.  Battery  status 

6.19 

bb.  Fuel  status 

6.19 

cc.  Vehicle  temperature 

4.97 

dd.  Other  information  requirement  (if  any) 

5.80 

6.  What  are  your  comments  on  infonnation  needed  for  tele-operating  a  vehicle? 

1 
1 
1 


Robots  surpassed  my  expectations. 

Great  experience.  Happy  I  could  contribute  my  input. 
Good  system. 
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Comments 


No.  of  Responses 


Picture  was  clear.  1 

Too  heavy  (transport  big/small).  1 

Hard  to  maneuver  the  robots  due  to  the  feedback  in  the  screens  being  so  choppy  and  1 
distorted. 

Improve  picture  and  ID  capability.  1 

Better  optics  and  training.  1 

At  least  one  camera  with  telescoping  potential.  1 

Signal  strength  is  the  biggest  contributor  to  overall  ability  to  discern  objects  in  1 

viewer.  Needs  a  very  good  camera  and  a  very  strong/consistent  signal. 

If  camera  had  swivel  capability  to  see  on  each  side  of  the  vehicle.  1 

Thermal  will  help  ID  people  and  running  vehicles.  1 

Maybe  an  instrument  panel/gauge  cluster  or  display  on  monitor  of  above  items.  1 

When  going  uphill,  it  is  more  difficult  to  see  close  to  the  front  of  the  vehicle.  1 

It  could  be  better  if  there  was  control  on  speed  on  the  robot.  1 

It  could  be  better  if  there  was  a  separate  control  on  the  camera  and  robot  (i.e.,  zoom  1 
in/out,  left/right/forward/rear,  etc.) 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
Only)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BELVOIR  VA  22060-5608 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  CS  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  ML  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MZ  A  DAVISON 
199  E  4TH  ST  STE  C  TECH  PARK  BLDG  2 
FT  LEONARD  WOOD  MO  65473-1949 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATTN  ATSACD 

ATTN  AMSRD  ARL  HR  ME  DR  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 


1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MM  DR  V  RICE-BERG 

BLDG  4011  RM  217 

1750  GREELEY  RD 

FT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARL  HRED  ARMC  FLD  ELMT 

ATTN  AMSRD  ARL  HR  MH  C  BURNS 
BLDG  1467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  348 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1 172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  WS  E  BLDG  3  RM  343 
NATICK  MA  01760-5020 

2  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MT  J  CHEN 

C  KORTENHAUS 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  ROOM  303 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
6501  E  11  MILE  RD  MAIL  STOP  284 
BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MF  MR  C  HERNANDEZ 
BLDG  3040  RM  220 
FORT  SILL  OK  73503-5600 

5  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

1  ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 

ABERDEEN  PROVING  GROUND 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  TECH  LIB 
BLDG  4600 


1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 
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